fix(ltx-2.5): the keyframe refusal named a FALSE reason, and a test pinned it (#644) - #666

…, and rescope W1/W2 (#608, #649) (#643)

FOLLOWING_AGENTS_PROTOCOL

The row said "Qwen-Coder XML, Mistral, pythonic, and remaining parsers", "Our
code: -", INVENTORIED, with a spec that was never written. All three parsers it
named by title were implemented, alongside 38 others. This is a RECORD REPAIR, not
an achievement: nothing new ships here.

Three counts were wrong and two of them cancelled, which is why they survived a
spec, a merge and a report. Ours is 41, not 40 -- the audit's regex used
[a-z0-9_]+ and silently dropped the hyphenated granite-20b-fc, while
test_detect.cpp:221 has asserted 41 since MODEL-MUSE-GLIMMER-W7. Upstream is 44,
not 43. And two of OUR names exist in no upstream registry at the pin (qwen3, a
local alias, and muse_glimmer, the off-pin unmerged-PR exception recorded in
porting-inventory.md §16). 44 - 5 = 39 shared, + 2 = 41. The old "40 = 43 - 3"
balanced only because two errors cancelled.

The five upstream-only names are confirmed, but the characterisation of them was
wrong in the dangerous direction -- "plain Python" reads as portable. Reading the
method bodies rather than the imports: GptOssToolParser is a declared stub whose
methods raise NotImplementedError and defer to Harmony; both Cohere classes
hard-ImportError without the out-of-tree cohere_melody package and delegate all
parsing to it. So only `inkling` is portable from vLLM source. W1/W2 are RESCOPED
accordingly -- one ordinary port, one from-scratch write gated on the upstream
test, and two names that each owe a recorded decision before any code.

State INVENTORIED -> PARTIAL, not ANCHOR-BACKFILL: that state asserts a missing
leaf spike and the spec is committed, while PARTIAL's "known to omit upstream
behavior" is exactly true of the five. Summary counts re-derived from the rows.

Reviewed twice. The first review FAILed it on four record findings -- a
docs/USAGE.md anchor that rotted during this branch's own rebase (#641 moved that
file by 572 lines), the portability characterisation, a defect flagged only in a
PR body, and a backwards precedent citation. All four repaired by a fresh
implementer; #649 filed for the stale TOOLS-CALLING-CORE counts. The scoped
re-review returned PASS with no findings.

Records only: 4 markdown files, no src/, no tests/, no checker weakened.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…FUSAL it carries (#607) (#642)

FOLLOWING_AGENTS_PROTOCOL

The multimodal seam map covered the input pipeline and the tower but not the
config that decides whether either runs. 43 of the 157 official recipes pass
--language-model-only and we reject it.

I filed #607 saying the flag "skips loading the vision encoder", implying a
boolean we could expose through an existing seam. The pinned oracle says
otherwise, and the correction changes the port. multimodal.py:78's own docstring:
"disables all multimodal inputs by setting all modality limits to 0. Equivalent
to setting --limit-mm-per-prompt to 0 for every modality." The encoder skip is a
CONSEQUENCE, at interfaces.py:293, which builds the tower inside no_init_weights
+ StageMissingLayer when ALL limits are zero; upstream's own docstring at :268
says so. Any route to zero limits gets it, so porting the boolean alone would
invent a path upstream does not have.

A fresh review then found the map still incomplete on the half that changes
BEHAVIOUR. validate_num_items (context.py:409-428) raises VLLMValidationError
from parse_mm_data (:461) and chat_utils.py:662, folded through allowed_mm_limits
(:392-405) -- so upstream --language-model-only REFUSES every multimodal request,
absent --enable-mm-embeds. That undercut my own claim that "L2 without L3 is
honest and shippable": as first written, L2 would have accepted an image request
upstream rejects, on the flag's PRIMARY observable axis, while my caveat guarded
only "frees VRAM". The refusal now sits in L1, where the limits' own semantics
belong, and L2's honesty is conditional on it rather than asserted.

Our baseline is nothing, not partial: no MultimodalConfig, no limit_per_prompt,
nothing gating tower construction, verified by searching for the behaviour rather
than the name. This is a port, and #607's title now says so.

Placed on ENG-MM-INPUT-PIPELINE rather than a new row -- that row is already "the
C++ mirror of vllm/multimodal/". Open PRs were diffed as well as issues before
writing; no branch touched either symbol.

One second-order finding worth the record: qwen3_next.py:325 proves the flag is
not purely a memory knob -- upstream uses it to select the FUSED QK-norm+RoPE+gate
path while ours is unconditionally fused. That is the serving-side twin of #414.

Spec plus two roadmap lines. No code, no row-state change. Reviewed twice; the
scoped re-review returned PASS.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Brings in `65e79eee5` (row 0, LTX25-PROMPT-ADALN) and `67e53e716`. Two keyed
records needed a hand resolution rather than the automatic three-way result,
because both sides had added a row under the SAME key:

* `.agents/roadmap_v1.md` — each branch appended its own `#644` row, which
  auto-merged into the issue being listed twice and `check-agent-record`
  failing. Resolved by taking one row per issue, naming both campaign rows
  (`LTX25-PROMPT-ADALN` and `LTX25-IMAGE-COND`) and both specs in it.
* `docs/FEATURES.md` — the LTX-2.5 lane row conflicted. Combined: the
  prompt-side AdaLN from main, the served image-conditioning arm from this
  branch. The combined text overran the 220-char cell and 600-char row entry
  budgets, so the forensics stay in the two specs and the row keeps the keyed
  projection, which is what that page is.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…eam (#553, #603, #651, #652)

main is RED on the DSR ratchet and has been since the LTX-2.5 landing: two kCUDA
sites at ltx2_video.cpp:531-551 put kcuda at 2 against a baseline of 0, so every
branch cut from main since inherits the red and each session re-derives that it
is pre-existing.

PROVENANCE, stated rather than claimed: this repair was written by a third
session and left unpushed - git ls-remote showed zero refs, so it was invisible
to gh pr list and to every ownership question any of us could ask. A peer found
it, verified it independently, and ceded it. I did not write it. It is adopted
here through the protocol to unblock main, with a fresh review by someone who is
neither its author nor its finder.

THE LOAD-BEARING CHECK. kcuda goes 2 -> 0 while check-device-leakage.py
(md5 9a4617a690e383d5a5c628ac3ec48111) and device-leakage-baseline.json
(md5 45ec32f4e08ab1c5ccf1f6ef236895b0) are BYTE-IDENTICAL to origin/main, and
git grep ltx2 over the checker is empty. So this is a repair, not a raised
threshold - the checker's own message forbids the latter in terms, and most
"green now" claims do not separate the two. vt_ifdef stays 32; total 34 -> 32.

It takes the platform-seam route the checker's message prefers (CurrentPlatform()
/ device_type()), not the ALLOWLIST escape. Only vt::Device{kCUDA, index} becomes
vt::Device{accelerator, index}; the CreateQueue/multi-GPU labelling comment at
:539-556 is untouched.

MUTATIONS, four, each anchor asserted count == 1 and the tree restored by md5:
  * revert the seam            -> RED, kcuda=3
  * --write-baseline after it  -> REFUSES: "REFUSING to write a HIGHER baseline"
  * hand-edit the baseline JSON to 35/3 -> checker PASSES. The ratchet's only
    defence against a raised baseline is a human noticing a 2-line diff, which
    is why md5 identity and not the exit code is the real check. Filed.
  * drop the accelerator == kCPU clause, keeping only TryGetBackend(...) == nullptr
    -> test_ltx2_video.cpp:386 FATAL, 502 -> 501 assertions. TryGetBackend(kCPU)
    is non-null, so without that clause a CPU-only build SILENTLY ACCEPTS
    device = 1. The guard is real and is now measured.

Gate: ctest -N registers 423, the serial run examined 423/423, 0 failed,
BUILD_EXIT=0, zero warnings. Registration is byte-identical to main, so the
denominator cannot have drifted. agent-preflight.sh on committed HEAD: all gates
green. test_cpu_x86_llamacpp_floor fails under load on unmodified main too and
passes at low load; environmental, reproduced on both sides.

Also corrects a defect in the original commit message: the M3a precedent is
src/vllm/entrypoints/model_loader.cpp:75-104, not model_executor/ - which is
ambiguous rather than merely unqualified, since a model_executor/model_loader/
directory also exists.

DELIBERATELY EXCLUDED: the author's live tip 8f26ee9 additionally carries an
unaligned-load change to ltx2_loader.cpp (#627) that arrived mid-review and was
never gated here. That file has four live rows on it right now, and its comment
claims fifteen casts across nine loaders while LoadUnaligned appears in four
files and six call sites. It wants its own scoped review rather than riding a
red-main repair.

Owed and filed, neither blocking: #659, the repair adopts M3a's seam but not its
companion supports_model_architecture guard, so a partial backend that could
decline by name now fails later in a kernel bind; #660, the kcuda bucket is the
token grep \bkCUDA\b, so minimax_h3_video.cpp:221-226 spelling it
static_cast<vt::DeviceType>(1) counts as zero and launders the same defect past
the same gate.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [ClaudeCode]

…inned it (#644)

Review findings on #657. Two blocking, four minor.

F1 — the last-frame keyframe refusal blamed `keyframes_abs_pos_embedding`, and
that is not what blocks it at pin `fd4ded7f`. Three anchors refute it:

  * `keyframe_cond.py:84-86` appends a SUPPLIED keyframe with `marked=False`,
    commented "given keyframe content is ordinary image guidance, not a
    generated keyframe slot, so it carries no keyframe marker";
  * its sole consumer is `transformer_args.py:42-43` (called once at `:269`),
    `mask = (keyframes_mask > 0); return hidden_states + mask * embedding` — so
    with `marked=False` it adds NOTHING, and porting it would not serve the arm;
  * the contrast was backwards. `tools.py:184` + `_first_frame_keyframes_mask`
    (`:186-195`) mark the target's FIRST latent frame unconditionally, so the
    tokens that actually reach the embedding are the ones the SERVED first-frame
    arm writes into — which the message called "needs none of that". That gap is
    real and is issue #658; it is not fixed here and nothing here contradicts it.

The named gate did not fire either: the refusal lives in `ltx2.cpp`, and
`Ltx2AdoptDeclaredDitParams` clears the flag on every real load under
`allow_unported_modules` because the shipped DiT does carry the tensor.

Worse, `test_ltx2_video.cpp` had been written to REQUIRE the wrong reason by
name. The message now names the token-append machinery — extended `positions`
(`keyframe_cond.py:46-59`), `update_attention_mask` (`:68-76`), extended
`clean_latent` / `denoise_mask` (`:79-82`) and `clear_conditioning` trimming back
(`tools.py:88-105`) — against a phase loop fixed at the target grid's token
count, and says why the first-frame arm needs none of it. The assertion now
requires THAT, and permits the refuted reason only where it is labelled as ruled
out and paired with #658. Local anchors are symbols, not same-file line numbers,
which is how the previous citation went stale. Sixth refusal in this campaign
whose stated reason was false; the header comment now says six and says why.

  RED   old message, new assertions:  4 failed, exit 1
  GREEN new message:                  32 cases / 565 assertions, exit 0

F2 — spec section 8.5 names "conditioning stage 1 only would let stage 2 re-noise
the pinned frame away" as the REASON the design conditions inside the phase loop,
and gated it with nothing. MEASURED: `if (wants_image && phase_index == 0)` left
the suite at 32 cases / 550 assertions / exit 0 — `image_digest` / `image_tokens`
are overwritten each phase and the only assertion was `image_tokens > 0`. This is
the shape of the obvious optimization (hoist the per-phase decode+encode out of
the loop), so it would have landed green.

The trace now pins the LAST phase's per-latent-frame token count, and a
`max_phase = 0` engine over the same request pins stage 1's, with the 4x relation
checked between two MEASURED values rather than two compile-time constants.

  RED   mutant: `CHECK( 1 == 4 )` twice plus the digest equality, 6 failed, exit 1
  GREEN 32 cases / 565 assertions, exit 0

F3 — nine mis-cited anchors, each re-verified against the pin before changing.
`blocks.py:576-580` is the TEARDOWN, so the conditioning-order claim now cites
`create_noised_state` (`helpers.py:428-445`) and the video-before-audio claim
cites `blocks.py:554-563`. Also corrected: `ImageConditioner` build `:985-986` /
build-and-free `:988-991`; `helpers.py:274-275` are parameter declarations, with
the per-stage h/w at `distilled.py:251, :255-256, :285-286`;
`model_configurator.py:41-43`→`:41-42`; decoder padding default `:90`→`:92`;
`reference_video_cond.py:74, 80`→`:36-37` applied at `:65-77`;
`reference_audio_cond.py:33`→`:34`; `detect_params` `:166-177`→`:166-179`.

F4 — `Ltx2VideoVaeEncoderKeyRules` claimed "rule for rule". Upstream's `SDOps`
(`loader/sd_ops.py:101-122`) is an `any()` admission over four matchings plus a
chained substring replace over three replacements; this port is one first-match
prefix pass, so its fourth rule is an identity carrying upstream's fourth
matching. The comment now says behaviourally equivalent, and names the case
where the two shapes would part.

F5 — `ParseEncoderBlocks` defaulted `num_layers` to 1 while upstream SUBSCRIPTS
`block_config["num_layers"]` (`video_vae.py:55`) and raises `KeyError`, and two
lines below it `multiplier`'s sentinel is deliberately strict. A `res_x` block
without `num_layers` is now refused; no other block kind reads the key, so
requiring it everywhere would have refused configs upstream loads, and both
halves are gated.

  RED   guard removed: "did NOT throw at all!", 1 failed, exit 1
  GREEN 15 cases / 223 assertions, exit 0

F6 — the three assertions comparing compile-time constants are left as they are,
correctly labelled documentation. Noted so the count is read honestly; the new
F2 assertions deliberately compare two measured values instead.

USAGE carried the same false keyframe reason and is corrected, with the
correction dated and #658 named so a reader can check it. USAGE also gains the
`res_x` / `num_layers` requirement. FEATURES keeps its keyed row within budget.

Gate, at this tree, CPU-only Release, CUDA=OFF:

  BUILD_EXIT=0, no ENOSPC, no BFD assertion
  ctest -N            424
  ctest -j 8          423/424; `test_engine_core_proc` is the documented
                      -j starvation and passes serially, exit 0
  test_ltx2_video     32 cases /  565 assertions, exit 0 (was 32 / 550)
  test_ltx2_image_cond 15 cases /  223 assertions, exit 0 (was 15 / 198)
  test_ltx2_vae       36 / 3039 | test_ltx2 35 / 2435 | test_ltx2_loader 26 / 4826
  test_ltx2_pipeline  37 / 2382 | test_ltx2_device 15 / 523 — all unchanged, exit 0

`LTX2_CHECKPOINT_ROOT` is unset here, so `test_ltx2_video`'s shipped-checkpoint
subcase SKIPPED and is not part of those counts.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

BUILD still marked MXFP4 as planned and said all GPU GGUF weights expand. The implemented Marlin and keep-quant paths make both claims stale.

STATUS also retained the old 30-entry registry count after the guarded public list grew to 37.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5 [Codex]

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>

…(W1, #332) (#473)

Implements the vt::Backend graph-capture seam on hipGraph (W1 of #332), mirroring
src/vt/cuda/cuda_backend.cu call for call.

Merged with the row's performance rationale REFUTED and recorded as such. W3
measured capture at +3.2% / +0.6% / -1.0%, not the ~2.2-3x §1 predicted, and the
spec's D7 plus the inline note in §1 now say so rather than leaving a live
rationale for the next agent to re-derive. That refutation was reported by the
contributor against their own interest, which is the behaviour this protocol
exists to produce.

Merged anyway on the seam argument, which is independent of the decode number:
graph capture had exactly one real implementation (CUDA), and a one-implementation
abstraction is unproven. hipGraph is the cheapest available second. Runtime cost
today is zero -- RocmPlatform does not override support_static_graph_mode(), so
nothing in the engine reaches the new code.

Review verified isolation three ways: every decode-graph call site ANDs
SupportsGraphCapture() with support_static_graph_mode(); rocm_backend.hip appears
zero times in a CPU build's compile_commands.json; and there is zero drift on
every touched file across the 194-commit gap. A mutation of the EndCaptureGraph
seam signature turns the new test red, so it genuinely guards seam drift on
machines with no AMD hardware.

Known-broken windows-msvc-* are the PR-only arm (#584), not this change.

Carried forward: #523's per-call hipMalloc/hipFree and hipStreamSynchronize are
illegal under hipGraph capture and must be reconciled before this capability
could ever be switched on.

…den is binary, so no golden-bearing PR could merge (#615) (#619)

Retires the fail-closed binary guard in scripts/check-pr-size.py (#615), and
classifies .agents/oracles/* (#668).

WHY THE BINARY GUARD GOES. It errored on any path git reports as binary, with no
exemption route. Parity goldens ARE binary, so no PR capturing an oracle golden
could merge -- blocking #431 and every future golden across the ROCm, Tenstorrent
and Nemotron lanes. The guard also contradicted this checker's own model (the
SITE_ASSET note says binaries "take the `asset` class the same way any other
shipped artwork does"), post-dated the golden precedent it rejected by one day
(450a1b6 2026-08-10 vs 971d550 2026-08-09), and a commit already on main
(5080983, 16 .npy files) trips it. A rule nothing in the tree can satisfy is
not enforcing a standard.

Explicit classification is KEPT and still refuses an unclassified binary, now
with a message naming the real defect instead of an unfixable property of the
file. No allowlist was added: a roster of blessed binary paths is a shared
must-write surface, which is the lock AGENTS.md forbids.

This removes no size rule because none was left -- the per-class budgets were
retired 2026-08-10. Nothing in this file measures a diff.

Reviewed by a fresh agent that did not write it: every claimed number
reproduced, all four sabotage mutations were caught (reintroduce the guard,
break classification, disable the evidence contract, break the PR-required
check), and seven adversarial commits confirmed the checker still refuses an
unclassified binary, a checker change without evidence, and a binary posing as
its own evidence. Its scope finding -- that goldens classify as `product`, so
this admits binaries to src/ and scripts/ too -- is recorded in the spec as an
accepted trade rather than argued away.

ALSO FIXES #668. The secondary-oracle registry landed .agents/oracles/<id>.md
with no pattern here, so all eight files were unclassified and classify_path
RAISED on unmodified main -- meaning any PR recording an oracle pin was refused
by a required check. Takes the same `procedure` class as .agents/specs/ and
.agents/claims/, which it is structurally identical to. Deliberately a pattern,
not a directory exemption: a non-.md or nested path there still fails closed.

device-leakage and sanitize-cpu are red on main itself and inherited by the
merge; this PR touches no C++. Both are being fixed separately.

Closes #615. Closes #668.

…ing lane (#634) (#637)

FOLLOWING_AGENTS_PROTOCOL

vLLM-Omni supports IndexTTS-2.5 and we had no record of it. Every audio path we
ship CONSUMES audio (Parakeet, Voxtral, `audio_processor.cpp`); nothing
synthesizes it. Rows both registered architectures --
`IndexTTS2TalkerForConditionalGeneration` (stage 0, ~0.8B GPT-2 AR talker) and
`IndexTTS2S2MelDecoder` (stage 1: EnhancedCodec -> S2Mel CFM/DiT -> BigVGAN,
22.05 kHz) -- at `registry.py` @ `bbe6ccc512a404a2df8c977ea29003002f2683e8`.

NO product code. This is a spec and its records; W1-W7 are all unstarted. Nothing
runs, no audio is generated, and the lane is not reachable from the ABI or the
server.

That unreachability is now SCOPE rather than a footnote, verified in the tree:
`include/vllm.h` is at ABI v18 with zero speech entry points, and
`audio/speech`/`audio/voices` have zero hits across `src/`, `include/` and
`examples/`. The spec mirrors the video lane that solved the same problem for a
generative modality -- a `SpeechEngine` seam like `VideoEngine`, ABI v19 with
`vllm_synthesize` named symmetrically to the existing `vllm_transcribe`, both
routes through that seam, the example as a thin ABI client -- and the seam must
admit the ~10 other omni TTS architectures, not just this one.

Three things pinned down rather than left to be discovered: reference audio is
MANDATORY (upstream has no text-only synthesis, so w2v-bert-2.0 + MaskGCT +
CAMPPlus are required surface); no token-exact e2e gate is possible (a seed drives
both AR sampling and per-request CFM noise), so the binding gate is per-stage
numerics vs the checkpoint's own remote code plus token-exact mel codes at fixed
seed/c1/fixed batch composition, with an ASR-round-trip ratchet that is explicitly
neither a correlation gate nor a count-based tolerance; and we mirror vLLM-Omni
even though it itself deviates from IndexTeam's `num_beams=3` default.

Two reuse claims were CORRECTED after LTX-2.5 merged: it generalized the VIDEO
engine, not the vocoder, and the BigVGAN 1-D core is already shared by H3 and
LTX-2.5 from `minimax_h3.h`. This lane would be the THIRD consumer, so W1 shrank
from "generalize" to "relocate to a neutral home".

Both rows land `INVENTORIED`, unclaimed and blocked on the UNPINNED vllm-omni
oracle. MODEL 370 -> 372, with mutation evidence for the two-architecture shape.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

… against the corrected bound (#496) (#675)

FOLLOWING_AGENTS_PROTOCOL

W2 of KERNEL-SSM-MAMBA (#496): the CUDA arm of vt::Mamba2ChunkScan,
vt::Mamba2StateUpdate and vt::RmsNormGatedGroup. Supersedes #566 and #592, which
had diverged -- one carried the evidence resolving the merge precondition, the
other the F1/F2 repair, and neither contained the other.

F1: the declared equivalence contract named the elementary functions as the ONLY
admitted source of device-vs-host divergence. Host C++ is pinned
-ffp-contract=off and nothing passes --fmad=false to nvcc, so acc += xv * bv
contracts on device and not on host. Stated model 3.5*K*u, contraction adds
~1.0*K*u, bound was 4*(K+2)*u -- provable only for K <= 18 while the driver
shapes run at K = 200. Repaired by carrying the term to 5*(K+2)*u with the
arithmetic shown in the header, all three test comments and 8.3. -fmad=false was
rejected deliberately: it is a per-TU flag on a header included by the hot GDN
TU. Repo-wide gap filed as #591.

F2: the header claimed the device kernels stay memory-safe under a contract
violation, but the validator checks metadata shape and dtype only. Two dropped
checks were memory-unsafe -- an out-of-bounds WRITE past a cudaMallocAsync
allocation and an out-of-bounds READ of initial_states -- and the stated reason
for omitting them (a D2H plus a stream sync) did not apply, since both values
are already in registers and the decode kernel does exactly that clamp for free.
Repaired with both the clamps and a narrowed claim enumerating all six dropped
checks. A third hole the review did not name was found and closed: seq_idx[0] < 0
indexes passed at chunk -1.

All four owed gates discharged by measurement, operator-run on the gate host,
each stamping its own lock-acquire time, load and disk. nvcc compile for sm_121a
clean with CUTLASS, FA2 and Marlin confirmed ENABLED in the configure log rather
than assumed. The three CUDA arms 12/2095, 10/5965, 12/3723 all SUCCESS.
compute-sanitizer memcheck ERROR SUMMARY 0 errors. And the 9-mutation re-sweep
against the MOVED bound: 9 of 9 CAUGHT.

The re-sweep is the item a reader would most likely have waved through, and M6
is why it could not be: it aborts at exit 134 while printing
"assertions: 2577 | 2577 passed | 0 failed" -- a clean assertions line on a
FAILING run -- and is caught only because the harness reads the exit code. Two
of the original eight mutations did not COMPILE under -Werror=all-warnings and
were being scored as caught; a mutation that will not build is a suite that
never ran.

test_minimax_h3 is attributed rather than waived. Reproduced standalone on an
idle box under the lock at TEST_EXIT=139, it is #486 with root cause #516,
signature-for-signature. The independent baseline that PASSED was
row/pool-device-key, the branch that FIXES #516, so the baseline carried a fix
and this branch does not carry a defect.

CI is REMOTE_UNVERIFIED, not green: every run on the predecessor branches ended
cancelled, including a repo-wide mass cancellation of 20 runs across 7 branches,
and cuda-fat-build never completed -- which is why the compile was run directly
on the gate host. Windows reds are the main baseline (#514, #584).

The row stays INVENTORIED. No lifecycle move, no measurement claimed, no
performance result: a host reference plus its device arm is not a speed number.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

… a merged pair (#517) (#684)

FOLLOWING_AGENTS_PROTOCOL

W2 of MODEL-TEXT-nemotron-h (#517): the non-gated relu^2 MoE expert.

NemotronH's expert has no gate half -- ckpt_names=("up_proj","down_proj",""),
where the empty third entry IS the absent gate. So this is not a new merged
pair. MergedGemmGroup describes N GEMMs sharing an operand with a fused
epilogue; with N = 1 there is nothing to merge and an arity-1 descriptor would
name a fusion that does not exist. It lands as the existing grouped projection
plus the activation we lacked, with the reasoning recorded next to the seam it
excludes, and a fresh review confirmed that is a correct reading rather than a
parallel path with a good story.

The finding that justifies the row: folding the routed scale into the
per-expert weight SURVIVED green in the first review, yet differs from upstream
bitwise on 13276 of 20000 rows -- and it is the most likely W4 mistake, because
Laguna legitimately does exactly that fold, legal there only because Laguna
passes no shared expert. Now pinned bitwise at 10 and 4 ULP separation.

Two conflicts on the re-merge. include/vt/ops.h had BOTH sides appending an
OpId before kCount -- kLtx2 from the LTX-2.5 campaign and kMoeRelu2 here --
resolved by union with main's entry first so no existing id shifts. And
op_provider.cpp auto-merged CLEANLY yet did not compile: main added an
exhaustive OpName switch with no default, documented as failing the build when
an OpId is appended without being named. That is the gate working exactly as
designed; the alternative is a new op with no canonical spelling that nothing
would have caught.

cuda_moe.cu compiled against the merged ops.h with real nvcc on the gate host,
BUILD_EXIT=0 WARNINGS=0 ENOSPC=0, closing the review's F1 -- whose original
justification, that neither file had changed since the last GPU compile, was
factually false: ops.h moved +180/-1 across six merges.

Post-merge: 0 diagnostics under -Werror, relu2 12/81 and router 14/941 both
SUCCESS, ten mutations caught. Squash-merged deliberately: the branch carries an
inherited trailer-less merge commit that a local --no-ff would put on main.

Row stays INVENTORIED. No lifecycle move, no measurement claimed.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…ech seam, family refusal (#634) (#681)

FOLLOWING_AGENTS_PROTOCOL

FOUR of the seven work items in .agents/specs/indextts-2-5.md. This does NOT make
IndexTTS-2.5 runnable: the lane detects its checkpoint and REFUSES, naming the
stages that are missing. No audio is synthesized by this change and none is
claimed.

W1 — the shared BigVGAN 1-D core moves out of `minimax_h3.h` into
`vllm::vocoder1d`. It lived there because H3 ported it first; LTX-2.5 reused
rather than copied it, and `ltx2_audio_vae.cpp:223-230` records why ("each copy
keeps its own green gate while the two audio VAEs drift apart"). IndexTTS-2.5 is
the THIRD consumer. 28 call sites retargeted; `MaterializeWeightNorm` stays with
H3, which is its only caller. Gated twice, because the failures differ: a
structural guard (one declaration site, one definition site, no retired
spellings) is the ONLY thing that can see a fork, since a fresh copy agrees on
the day it is made; and 10 cases of HAND-COMPUTED arithmetic, not values captured
from the implementation, which would pass just as happily if the helper were
uniformly wrong. `test_minimax_h3` 79/79 and `test_ltx2_vae` 36/36 prove the move
is behaviour-preserving.

W2 — the GPT-2 talker backbone, ported from `gpt2.py` @ 555967922 with the
reference checkout verified at that SHA before any line was cited. The oracle is
a TP=1 restatement (upstream's module needs a KV cache, a device and an
initialized distributed environment), the technique gen-minimax-h3-goldens.py
already uses. Hidden states within 2e-5, logits within 5e-5, argmax TOKEN-EXACT.
The Conv1D transpose and causality each get their own case because both fail
silently: goldens are emitted in upstream [in, out] orientation so the loader
must transpose, and a non-causal model stays fluent.

W6a — the `SpeechEngine` seam, mirroring `VideoEngine`, shaped for a FAMILY
because vLLM-Omni carries ~10 further TTS architectures behind this one.

Family registration — an IndexTTS-2.5 checkpoint now resolves to a named family
and refuses with the missing pieces spelled out, which is the AGENTS.md contract
for an unimplemented arm.

Also records the MEASURED component inventory (gpt 17,171 LOC, s2mel 15,011,
utils 18,265, codec 1,930) and one correction that changes the plan: the talker
is built with `spk_cond_mode="campplus"`, so CAMPPlus is UPSTREAM of it and W3
cannot be deferred.

STILL OPEN: W3 (w2v-bert-2.0, MaskGCT, CAMPPlus), W4 (EnhancedCodec, S2Mel), W5
(compose), W6b (routes + ABI). No e2e render, and any parity claim additionally
needs the vLLM-Omni oracle, still UNPINNED (#633).

Verified locally: six suites green (10/4/7/4/79/36 cases) plus the structural
guard; check-test-registration, check-agent-record, check-doc-checkpoint and
check-pr-size green. Every gate was mutation-proven, and TWICE a mutation's build
failed under -Werror so the old binary re-ran and printed SUCCESS; compile status
is now checked alongside every mutation result.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…tream itself (#634) (#689)

FOLLOWING_AGENTS_PROTOCOL

First increment of W3. Does NOT make IndexTTS-2.5 runnable; the lane still
refuses by name.

CAMPPlus is the speaker-style encoder on the mandatory reference-audio path and
sits UPSTREAM of stage 0 — the talker is built with `spk_cond_mode="campplus"`
(infer_v2_5.py:138) and consumes its style vector.

A STRONGER ORACLE THAN W2's: `campplus/layers.py` has no vllm dependency, so the
generator loads it BY FILE PATH and executes the REAL classes. W2 needed a TP=1
restatement because upstream wanted a KV cache and a distributed environment;
here there is none, so there is no risk of being faithful to the wrong thing.

Four primitives, each a silent-failure mode: StatsPool (UNBIASED N-1 std; the
biased form differs ~0.2% at T=250, reads as noise, moves a style vector),
BatchNorm1dEval (RUNNING statistics; batch statistics still normalize and are a
different model), SegPooling (ceil_mode expand-then-truncate, goldens at T=250 so
the trailing PARTIAL segment is exercised) and CamLayer. 4 cases / 8 assertions,
all within 1e-5 of upstream.

MUTATION EVIDENCE with compile status alongside every result: biased std, floor
instead of ceil_mode, and dropping the global-mean term each go RED at
compile_err=0; restored tree returns to SUCCESS. The compile check is there
because this campaign already produced TWO false readings where a mutation failed
to build under -Werror and the old binary re-ran printing SUCCESS. A killed
mutation run also left the tree dirty mid-sequence — the file is untracked so
`git diff` could not see it, and it was caught by grepping for the mutated line.
Restoration is verified by content now, not assumed.

STILL OPEN: CAMPPlus's FCM 2-D front end and 52-layer CAMDenseTDNN stack; then
w2v-bert-2.0 and EnhancedCodec; then W4 S2Mel, W5 compose, W6b routes/ABI. No
e2e render, and any parity claim needs the still-UNPINNED oracle (#633).

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

… gate caught (#634) (#690)

FOLLOWING_AGENTS_PROTOCOL

Adds TransitLayer, DenseLayer, CAMDenseTDNNLayer and CAMDenseTDNNBlock on top of
the primitives already on main, gated against `campplus/layers.py` executed
DIRECTLY (no restatement). 8 cases / 16 assertions, within 1e-5.

A REAL DEFECT, found by the gate rather than review: upstream's final dense uses
`config_str='batchnorm_'`, which get_nonlinear expands to a SINGLE batchnorm with
affine=false and NO relu. The first implementation applied one unconditionally,
clamping every negative component of the style vector to zero — a plausible
embedding from the wrong model, invisible to any shape check, and the talker
consumes exactly that vector.

Two ordering traps also gated, each producing a correctly-shaped wrong tensor:
TransitLayer applies its nonlinear BEFORE the projection while DenseLayer applies
it AFTER, and the block's cat([x, layer(x)], dim=1) growth means appending in the
wrong order still yields the right shape.

MUTATION EVIDENCE with compile status per result: forced relu, prepend-instead-of-
append, and swapped transit order each RED at compile_err=0. The relu mutation's
first attempt reported compile_err=1 and a meaningless SUCCESS — the third time
this campaign that a mutation failed to build under -Werror and the old binary
re-ran, which is why compile status is now reported beside every result.

REMAINING: CAMPPlus's FCM 2-D front end and forward assembly; then w2v-bert-2.0
and EnhancedCodec; then W4 S2Mel, W5 compose, W6b. No e2e render.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…racle is not vLLM (#672) (#679)

* spec(MODEL-MUSIC-MUSIC3): scope MiniMax-Music3, the first row whose oracle is not vLLM (#672)

MiniMaxAI/MiniMax-Music3 is text-to-music: lyrics plus a structured description
in, a multi-minute stereo song out. It is absent from the pinned vLLM (no source
files; the registry carries only the MiniMax M2/M3 TEXT architectures), absent
from vLLM `main`, and absent from vllm-omni. So it is the first row to exercise
the fallback-oracle rule added in #647, and its primary oracle is an OPEN
diffusers PR.

W0 ONLY. Spec committed before implementation, per AGENTS.md. No src/, include/
or tests/ change is in scope here; W1 onward are dispatched to fresh implementers
from this spec rather than written in the coordinating session.

GEOMETRY MEASURED, NOT INFERRED. Every component config.json read, plus the
safetensors headers by HTTP range request -- the same no-payload technique LTX-2.5
L0 used. What that corrected, twice:

  * transformer is 231 tensors / 1.240B params per shard, ALL F32. The card's
    "2.4B" is right and the 9.73 GB on disk is fp32 storage, NOT a 4.9B bf16
    model. Reading the byte count alone would have sized it at double.
  * condition_encoder has FOUR tensors -- layer_scale, layer_weight_logits and a
    proj. It is a learned weighted mix over 8 LLM hidden layers, not the encoder
    tower its name suggests.

Also measured: rvq_depth_decoder 47 tensors / 0.646B BF16, vocoder 121 / 0.054B
F32 carrying weight_g/weight_v, so weight-norm must be folded at load.

THE LARGEST BRICK IS ALREADY BUILT. language_model is `Qwen3ForCausalLM` at 36L /
4096 / 32 heads / 8 KV / head_dim 128 -- our LANDED architecture exactly --
retrained on a 200 000-entry music vocabulary. Vocabulary size is a config value,
not an architecture change.

TWO PACKAGINGS, ONE SET OF WEIGHTS, and that had to be established rather than
assumed, because the two oracles disagree about which one they serve. SGLang-Omni
loads ONLY the native arm: checkpoint.py:35-56 resolves qwen_7B/qwen_7B,
flowmatching_vae.pth and dav.pth, and pulls the RVQ decoder out of the Qwen shards
by the model.audio_decoder. prefix. diffusers loads the six modular components.
convert_minimax_music3_to_diffusers.py@c6da9936 :29-38,47-189 settles it -- load
the three native artefacts, rename tensors, save. A pure re-layout, no numerical
step. Therefore the diffusers arm is the port target (~28.5 GB resident, no
pickle, no trust_remote_code, per-component gateable) AND SGLang-Omni stays a
valid e2e cross-check instead of an incomparable second model. W1 verifies that
equality against actual tensors rather than inheriting it from this reading.

THE DTYPE POLARITY IS UPSTREAM'S. The conversion script defaults --dtype float32
(:267), gives transformer/condition_encoder/vocoder that dtype (:208-211) and
forces the RVQ decoder to bf16 regardless (:214) -- matching the measured headers
exactly, and SGLang-Omni's README confirms both placements run the acoustic stage
in FP32. So fp32 there is a resolved upstream choice, not the too-wide accident
AGENTS.md warns a token gate cannot see. It is mirrored, and annotated as this.

ONE CONTRADICTION LEFT OPEN ON PURPOSE. The model card and SGLang-Omni's README
say 32 kHz stereo; every config says 44100 (vocoder sampling_rate, and
condition_encoder output_sampling_rate with hop 512). Both cannot be right and the
difference is a resample -- which no tensor-parity check on the latents would
catch, so it is exactly the class of defect that survives a green gate. W0
resolves it from source before ANY waveform golden is captured. It is recorded as
unresolved rather than guessed.

ORACLES PINNED. diffusers advances from UNPINNED to the PR #14456 head
c6da9936 -- the head SHA and not the branch name, because
huggingface:minimax-music3-integration can be rebased under us and "whatever the
branch was that day" is not reproducible. SGLang-Omni gets its OWN record at
748a0b43: it is a third repository with its own cadence and its own registry, and
folding it into sglang.md would have hidden that a model can be in one and in
neither of the others. Both stay gateable = no -- read, never executed here --
which is the honest state and what W0 exists to change.

GATE SHAPE, stated up front because conflating the halves is the failure mode:
the LLM half emits discrete RVQ codes and gets a real token-exact gate; the
acoustic half is a flow-matching loop with no logits and no sampler, so it gets
per-stage tensor parity at fixed seed and reduced dimensions, following H3. A
correlation coefficient is NOT a gate there -- Pearson is scale-invariant, so a
uniformly scaled latent passes it while sounding wrong.

Records: MODEL row ratchet 370 -> 371 (the test that mutates it to expected-1
keeps the bump load-bearing), checklist entry + rollup SPIKE 8 -> 9, the 11
non-pin architectures become 12, roadmap issue table, and a claim file.

Gate: scripts/agent-preflight.sh -- all checkers and all suites ok EXCEPT
test_cpu_x86_llamacpp_floor, a contention-sensitive harness that failed
identically (NO_QUIET_WINDOW, exit 4) on an unmodified tree at base SHA cefacd2
under load average 35. Pre-existing and environmental, verified rather than
assumed.

Issue: #672, #647

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

* spec(MODEL-MUSIC-MUSIC3): the 32 kHz / 44100 split is a STAGE BOUNDARY, not a contradiction (#672)

W0 owed this from source before any waveform golden could be captured. Resolved,
and the answer is that both numbers were right about different points in the
pipeline.

THE VOCODER NATIVELY EMITS 44100 Hz STEREO, and it is derivable rather than merely
declared -- which matters, because a declared sample rate in a config is exactly
the kind of value that goes stale without anything noticing. The condition
encoder's output_sampling_rate 44100 / output_hop_length 512 set a latent frame
rate of 86.133 Hz (condition_embedder_minimax_music3.py:40-41), the decoder
applies one ConvTranspose1d per upsampling_ratios entry for 8*8*4*2 = 512x
(minimax_music3_vocoder.py:84,92-95), and 86.133 * 512 = 44100. The declared
sampling_rate matches its own convolution stack. SGLang-Omni's independent
implementation lands on the identical numbers (dav.py:94,115), which is the
strongest form of confirmation available here: two codebases that were not copied
from each other.

DIFFUSERS RETURNS THAT UNRESAMPLED (modular_pipeline.py:32-36, decoders.py:84-92).
SGLANG-OMNI'S SERVER RESAMPLES 44100 -> 32000 ON DELIVERY (constants.py:18-19,
acoustic.py:55-58,422-431). So "32 kHz stereo" in the model card and in
SGLang-Omni's README is neither wrong nor stale: it is the reference SERVER's
delivery rate. diffusers' own docs state the split in as many words.

DECISION: goldens are captured at 44100 stereo -- the model's native generative
rate, resample-free, and what the primary oracle hands the caller. The 32 kHz form
is a downstream delivery transform, gated separately if SGLang-Omni byte parity is
ever wanted.

That transform is NOT free, and the spec says so rather than leaving it to be
discovered: acoustic.py:58 passes no lowpass_filter_width, rolloff or
resampling_method, so reproducing those bytes means reproducing torchaudio's
DEFAULT sinc filter, not converting 44.1 -> 32 by any correct method. And the
reason this had to be settled before the first golden, not after: a latent-tensor
parity check sits entirely upstream of the resample call and cannot see the
difference. A golden captured at the wrong rate would be wrong by a resample while
every tensor gate stayed green -- the class of defect that survives a green gate,
which is the one this project spends its evidence discipline on.

Also corrected: the 24000/960 pair in condition_encoder/config.json is the AR
stage's 25 Hz frame rate, unrelated to output. It reads like a third sample rate
and is not one.

W0 now owes only the oracle stand-up. The §1.1 stop condition is removed because
it is discharged; the W5 row names 44100 stereo instead of "the resolved rate".

Issue: #672

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…AND aarch64 (#634) (#701)

FOLLOWING_AGENTS_PROTOCOL

CAMPPlus's head treats the [1, feat_dim, T] spectrogram as a 2-D image, so this
adds genuine Conv2d/BatchNorm2d and BasicResBlock. Gated against
`campplus/layers.py` executed directly: 9 cases / 19 assertions, within 1e-5.

THE TRAP THIS PINS: BasicResBlock strides `(stride, 1)` (layers.py:224) —
FREQUENCY only, TIME untouched. Striding both still yields a well-formed tensor
at half the frame rate, which every later layer accepts. Goldens confirm
16x20 -> 8x20 empirically, and the case asserts the WIDTH is unchanged alongside
the values, so a symmetric stride cannot pass by matching numbers on a smaller
tensor.

CROSS-ARCHITECTURE VERIFICATION: the suite was rebuilt from source and re-run on
kairos-4db2 (NVIDIA Thor, aarch64, g++ 13.3, no cmake — compiled directly) and
reports the same 9/9 and 19/19. The goldens were generated on x86_64 with torch,
so this establishes the port's numerics are not x86-specific and no
FMA-contraction difference breaks the 1e-5 tolerance on another arch.

MUTATION EVIDENCE with compile status per result: striding both axes, dropping
the post-residual relu, and dropping the residual add each RED at compile_err=0;
restored tree SUCCESS, verified by grepping the restored line.

Generator's hard-coded upstream path generalized to $CAMPPLUS_LAYERS.

REMAINING in CAMPPlus: the forward assembly over 3 blocks of 12/24/16 layers.
Then w2v-bert-2.0, EnhancedCodec, S2Mel, compose, routes/ABI. NO e2e render and
none claimed; the lane still refuses by name.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…l what is not paired

test(#669): pin that the token budget SPLITS a prefill wave, and label what is not paired

A measured claim of mine was wrong and this row is the disproof. I measured our
35B prefill as INVARIANT to --max-num-batched-tokens (0.6814s at 8192 vs 0.6828s
at 2048, c4) while vLLM's swung 30%, and concluded we were failing to chunk.

We are not. The budget reaches the scheduler -- chain verified hop by hop from
server_main.cpp:400,806 through model_loader.cpp:626-641,704-717,1051-1058 to
scheduler.cpp:233-234,465 -- and a composition probe shows chunking happening:
budget 8192 gives one 4096-token step; budget 2048 gives 2048 / 2048 / 5, with
the clamp firing mid-prompt (a=1, b=1, c=1024, d=1022). We are invariant because
1x4096 and 2x2048 are THE SAME TOTAL PREFILL WORK.

The encoder-decoder disable cannot fire, for a stronger reason than "the
condition is false": SchedulerConfig::PostInit has ZERO call sites outside its
own test, so it is never invoked on the engine path at all.

No timing gate can separate "chunking works and the totals are equal" from
"chunking never happened" -- which is why the test pins the COMPOSITION. Its
1022-token remainder is load-bearing: a reviewer's mutation that skips the
running loop keeps total_num_scheduled_tokens at exactly 2048 and is caught only
by that value. Six mutations RED, three of them the reviewer's own.

WHAT IS ESTABLISHED, and what is not:
  * intra-pin, paired: the pin's own 4096-token forward costs 42% more wall
    clock than two 2048s for identical work (0.8371/0.8147/0.8148 vs
    0.5789/0.5789/0.5774; spreads 2.7%/0.3%; n=3; interleaved; clock pinned per
    leg; boot_id logged; oracle asserted). Our invariance likewise (1.4%/1.0%).
    So the 1.21x we appeared to win at 8192/c4 is us beating an upstream
    pathology, not headroom.
  * cross-arm: NOT paired and not differenceable. `ours` came from the
    chunkprobe barrier-client with --num-blocks 2048; `pin` from mnbt.sh's curl
    spawn-loop with --gpu-memory-utilization 0.6; never interleaved with each
    other. The residual "+3.0% for us vs +11% for the pin" is a HYPOTHESIS.
  * --language-model-only was PIN-ONLY, not a shared condition.
  * the ours@8192/c1 cell is a mean of TWO legs; its rep-1 leg recorded 6135.86s
    -- an artifact of a wedged harness whose resume ledger re-imported it, which
    the reducer correctly flagged NOT ESTABLISHED on a 34902x spread.

Two upstream anchors corrected: the cudagraph cap is 64, not 512 (both budgets
still tie), and "upstream's flat 2048" is a pydantic TESTING default that
EngineArgs._set_default_args overrides to 8192/16384 on a >=70 GiB GPU. GB10 is
one, so our MoE-8192 may BE upstream's effective default -- the framing that
choosing it flattered us against a "true" 2048 bar is withdrawn.

Filed, not fixed: #682, the sweep gave our arm 8,192 tokens of KV against the
pin's 1,819,368 -- a 222x asymmetry that suppressed the very batching being
varied. Provisioned, our arm ran slightly FASTER, so the 0.85x is conservative
rather than flattering, but it is owed a re-measure before being banked.

No product change: the scheduler already mirrors upstream.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [ClaudeCode]

…n exposed (#634) (#706)

FOLLOWING_AGENTS_PROTOCOL

Completes CAMPPlus, the speaker-style encoder the talker consumes
(spk_cond_mode="campplus"). FCM 2-D head -> TDNN -> three CAMDenseTDNN blocks
(12/24/16) -> transits -> StatsPool -> dense, 12 cases / 27 assertions against the
real upstream class executed directly.

Weights are keyed by upstream state_dict NAME with the 815-tensor MANIFEST
emitted into the goldens, so a tensor one side builds and the other does not is a
failure; a missing tensor throws BY NAME, gated.

A GATE HOLE, found by mutation and closed: changing the TDNN head's padding 2->1
compiled cleanly and the suite still PASSED, because it changes the frame count
20 -> 19 and StatsPool averages over time, so the pooled embedding absorbs it.
Closed by tapping the post-TDNN activation upstream and exposing it via
ForwardTrace — channels, frame count and values are gated where pooling cannot
hide them. Both mutations now RED at compile_err=0.

Also caught for the fourth time this campaign: a failed -Werror build re-ran the
previous binary and printed SUCCESS, spotted because the CASE COUNT did not
change while the compile-error count was 1.

REMAINING: w2v-bert-2.0, EnhancedCodec, S2Mel, compose, routes/ABI. No e2e claim.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

) (#707)

FOLLOWING_AGENTS_PROTOCOL

Starts w2v-bert-2.0, the semantic front end IndexTTS-2.5 runs over the reference
clip (infer_v2_5.py:174). Ports the parts that make the encoder a CONFORMER: the
macaron feed-forward and the depthwise convolution module. 3 cases / 181
assertions, within 1e-5.

ORACLE: `transformers` executed DIRECTLY — the class IndexTTS itself
instantiates, so no restatement. Admissible under AGENTS.md §"When vLLM has no
implementation": a model's own reference implementation.

THE TRAP: the conv module pads (kernel-1, 0) — LEFT ONLY, so a frame never sees
the future. A symmetric pad yields an identically shaped output of the same
length, so neither a shape nor a length check finds it. Gated on upstream values
AND by an independent causality probe: perturbing the LAST frame must leave every
earlier output bit-identical while the last one moves.

Also pinned: swish not gelu, and the GLU gate orientation.

MUTATION EVIDENCE with compile status: symmetric pad, gelu-for-swish and swapped
GLU halves each RED at compile_err=0. The gelu mutation's first attempt did not
APPLY (anchor count 0, Swish being a one-liner) and printed a meaningless
SUCCESS; the anchor assertion caught it.

DEFERRED deliberately: relative-key self-attention, so it lands whole. Then
EnhancedCodec, S2Mel, talker head, compose, routes/ABI. No e2e claim.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…they carry — L1 (#607)

FOLLOWING_AGENTS_PROTOCOL

43 of the 157 official vLLM recipes pass --language-model-only and we rejected it.
L1 ports the mechanism behind that flag: per-modality input limits, and the
REFUSAL that makes a limit a limit.

Precedence mirrored literally from multimodal.py:321-336 -- language_model_only
returns 0 for EVERY modality BEFORE the map is consulted, else the map entry, else
999. The case that distinguishes the orderings is a modality absent from the map,
and it is pinned. The refusal mirrors context.py:392-405 (the min() fold),
:409-428 (message and its conditional hint, character-identical including the
backticks), and both call sites, with the enable_mm_embeds escape in both
spellings.

The judgement call worth recording: the refusal throws vllm::v1::InputValidationError,
MOVED into its own header rather than a fresh multimodal exception class. api_server.cpp
already maps that type to HTTP 400, mirroring upstream's ValueError -> BadRequestError.
A bespoke class would have passed its own tests and then surfaced a too-many-images
request as a 500. The review confirmed exactly one definition, no include cycle, and
all 17 TUs seeing the identical type -- and stated the honest limit: the 400 path is
unproven END TO END until L2 wires a call site, so L2's reviewer must treat that as
an owed gate rather than an inherited pass.

Scope is L1 only, verified clean in both directions: no serve flags, no ABI field,
no tower gating, no kernel gate -- and the refusal was NOT deferred out.
chat_mm.cpp is untouched, so its first-image-and-break truncation stands, filed as
#686 and named in the PR body and USAGE.md rather than silently left. Wiring it is
L2, which is what makes #686 the reason L2 exists.

Reviewed on aarch64 (a CI-supported lane; the dev box was at 100% disk and load
~100 and was not used). Clean build 1262/1262 under -Werror with zero warnings;
ctest 424/425, the sole failure test_async_llm passing 1/1 serially. 8 mutations
run, 8 caught, none survived. The reviewer discarded its own first sweep after
finding cp -a preserved mtimes and ninja had skipped a rebuild, so one result came
from a stale binary carrying the previous mutation.

The implementer's self-declared coverage gap is real and was judged correct
layering rather than a hole: the precedence is MultiModalConfig's own contract and
its own suite owns it. docs/USAGE.md was forced by check-doc-checkpoint classifying
any include/vllm/ edit as user-facing; the entry states no request is limited or
refused today, so a reader learns strictly less capability than exists.

Row stays READY: no lifecycle move, L2-L4 owed.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…engine (#672) (#700)

`multimodal::SpeechEngine` did not exist when this spec was written. The
IndexTTS-2.5 lane landed it on 2026-08-13, along with `Vocoder1D`, while W0 was in
review. AGENTS.md is explicit that a capability not reachable through the shared
surface is not done, and that a seam is EXTENDED rather than forked -- so the
scope changes before an implementer builds the wrong thing, not after.

IT FITS BETTER THAN IT LOOKS. `SpeechResult` already carries `channels`, and
already documents `sample_rate` as "the family's native rate ... rather than a
resampled one, so the caller decides whether to resample". That is exactly §1.1's
finding arrived at independently: 44100 stereo is the model's rate, and
SGLang-Omni's 32 kHz is a caller-side transform. `requires_reference_audio()`
exists so a server can refuse before staging weights; Music3 returns false where
IndexTTS-2 returns true, which is the difference between a voice cloner and a
text-to-music model expressed in the seam rather than around it.

THE ONE GENUINE GAP IS `SpeechGenParams`, which carries a single `text` field
because IndexTTS-2 synthesises one utterance. Music3 takes TWO distinct inputs --
lyrics with section tags, and a structured music description -- plus generation
controls (duration/frames, denoise steps, CFG). Packing both into `text` behind a
separator would be a private protocol inside a shared struct, which is precisely
the fork the rule exists to prevent. W6 extends the struct ADDITIVELY and leaves
IndexTTS-2.5 byte-identical: a field an existing family ignores costs it nothing,
while a second parallel params struct costs every future family a choice. If the
extension cannot be made additive, that is a NEEDS_DECISION, not a fork.

ALSO RECORDED: `SpeechEngine` is not on the ABI yet. include/vllm.h is at v18 and
exposes the video engine but no `vllm_speech_*` surface, and no open PR adds one
(checked the 15 open PRs). So W6 owns that surface, its version bump, and the
example HTTP server as a thin client of it -- and the W5 row now names `Vocoder1D`
as the thing to route through rather than leaving "the vocoder" to be invented.

W5 and W6 both gain the same obligation as a done-when: H3 and IndexTTS-2.5
behaviour stays byte-identical. Touching a seam another lane owns is allowed;
moving its numerics silently is not.

Issue: #672

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…#709)

FOLLOWING_AGENTS_PROTOCOL

Ports Wav2Vec2BertSelfAttention with position_embeddings_type="relative_key",
gated against `transformers` executed directly. 5 cases / 184 assertions, within
1e-5.

THREE THINGS THAT FAIL QUIETLY, each mutated:

THE CLAMP IS ASYMMETRIC (64 left, 8 right). A symmetric clamp still produces
well-formed attention and diverges only for keys further ahead than right_max.
The goldens use T=12 precisely so the RIGHT clamp bites; at T<=9 the mutation
would pass.

THE DISTANCE IS KEY MINUS QUERY. The reverse is a mirrored bias that still
attends and still trains.

THE RELATIVE TERM IS SCALED SEPARATELY by 1/sqrt(d), after the scores already
were.

Also pinned: the encoder is NOT causal — a case perturbs the last frame and
requires the FIRST output to move, the opposite of the conv module's guarantee
gated in the same suite.

MUTATION EVIDENCE with compile status: sign flip and missing scale RED at
compile_err=0. The clamp mutation first read compile_err=1 (right_max unused
under -Werror) and printed a void SUCCESS — the fifth occurrence this campaign.

Generator fix: self_attn is called with KEYWORD args so a positional
forward_pre_hook sees an empty tuple; the preceding layer_norm output is tapped.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…er (#634) (#710)

FOLLOWING_AGENTS_PROTOCOL

Composes the three gated parts into Wav2Vec2BertEncoderLayer and gates the WHOLE
block against `transformers` executed directly. 6 cases / 186 assertions, within
2e-5.

WHY THE ASSEMBLY NEEDS ITS OWN CASE: the macaron 0.5 factors live in the LAYER,
not in FeedForward, so every piecewise case already on main passes with or
without them. Green parts do not imply a green whole.

The conv module's absent pre-norm is the same trap in reverse: it normalizes
internally, so adding one here double-normalizes and still produces a well-formed
block.

MUTATION EVIDENCE with compile status: dropping the first macaron 0.5,
double-normalizing before the conv module, and dropping the final layer norm each
RED at compile_err=0; restored tree SUCCESS.

The w2v-bert ENCODER LAYER is complete. Remaining: model-level wrapper (feature
projection + layer stack), then EnhancedCodec, talker head, length regulator,
S2Mel. No e2e claim.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…#712)

FOLLOWING_AGENTS_PROTOCOL

Completes w2v-bert-2.0, the stage that turns the reference clip into the features
EnhancedCodec quantizes. 8 cases / 190 assertions against `transformers` executed
directly.

FeatureProjection normalizes THEN projects. The encoder is the layer stack and
NOTHING else: embed_positions is None for relative_key, and there is NO final
layer norm after the stack — the trap worth naming, since most encoders end with
one and adding it yields a correctly shaped tensor only a value comparison can
reject.

MUTATION EVIDENCE with compile status: adding a final norm, and projecting before
normalizing, each RED at compile_err=0.

PROCESS NOTE: this change's worktree lost a ref-lock race against a concurrent
session, so `cd` failed and the chained commands ran in the SHARED CHECKOUT,
editing two tracked files. Caught on the next command and reverted; checkout
clean. A failed `cd` does not stop a chain from running wherever the shell
already was.

Pipeline: w2v-bert DONE, CAMPPlus DONE, vocoder core DONE, GPT-2 backbone DONE.
Remaining: EnhancedCodec, talker head, length regulator, S2Mel, compose,
routes/ABI. No e2e claim.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…d no registry face (#608 W1)

FOLLOWING_AGENTS_PROTOCOL

W1 was scoped as "port a parser". The Inkling dialect was ALREADY ported and
oracle-gated -- src/vllm/parser/inkling.cpp, inkling_config() with its arg carver,
get_parser_engine("inkling"), golden scenarios in test_parser_engine_assembly.
What was missing was the tool-parser REGISTRY FACE: get_tool_parser("inkling")
returned nullptr, so --tool-call-parser inkling threw at startup and a fully
ported dialect was unreachable. 41 -> 42 names, 37 -> 38 families.

The change is ParserEngineToolAdapter (the TOOL half of upstream's make_adapters,
twin of the reasoning adapter already shipped) plus InklingEngineToolParser, the
factory branch and the tool_parser_names() entry together, and
ParserRequestFromChatCompletion moved beside ParserRequest so the adapter and the
serving path share one projection instead of two copies. The mutation that drops
tool_choice from that projection reds 20 cases -- that is what gates the move.

No detect.cpp marker row, for a reason no other excluded family has: the table
matches a CHAT TEMPLATE string, and Inkling has no jinja template at the pin
(rendering is vllm/renderers/inkling_encoding.py). Its marker collides with
nothing; a row would look harmless while being unreachable.

A secondary-oracle premise was REFUSED rather than followed. I briefed that
GptOssToolParser's stub meant vLLM implements nothing for gpt-oss, so SGLang was
admissible under the new registry rule. It is not: vllm/parser/harmony.py is a
358-line vLLM-owned HarmonyParser(DelegatingParser) with its own test, and the
stub delegates INSIDE vLLM. Following my brief would have consulted a secondary
oracle where the primary speaks. Recorded in the spec so nobody re-runs the check.

Reviewed, FAILed on the record, repaired, and the corrections are larger than the
original claim. "Upstream never constructs the adapter" was false --
tests/parser/engine/test_inkling.py:487 TestRegisteredAdapters does exactly that --
and that sentence was load-bearing, because it justified never looking outside two
test classes. Four more cases ported, one declined with its reason; the split is
22 cases, 19 PORTED / 3 AUTHORED, not the 15/18 first reported. The authored
CONTENT-seed case survives but its rationale was also wrong: the seed lives only
in extract_tool_calls_streaming, not in the delegating non-streaming path.

Two gaps left OPEN and recorded rather than papered over. finish_streaming() is
functionally ungated -- mutating it to return nullopt leaves all 22 cases green;
the repairer authored a streaming twin to try to move it, it did not, and the case
was DELETED rather than ship a false rationale. And README's counts are stale by
two waves, but check-doc-checkpoint REFUSES the fix ("Co-edited public projections
never justify README churn") and refuses a README-only fix too, so #704 needs a
landing-source change or an argued exception. #703 records that
--reasoning-parser inkling still throws while the tool parser now resolves.

Verified on aarch64 (kairos-4db2, CPU-only; the dev box was at 98% disk). Clean
build 0 warnings under -Werror, ctest 432/433 with the one failure passing
serially, test_tool_parser_inkling 22/22 133/133.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…c codes (#634) (#713)

FOLLOWING_AGENTS_PROTOCOL

Ports Amphion's FactorizedVectorQuantize (use_l2_normlize=True) — the
`semantic_codec.quantize(...)` of infer_v2_5.py:293, whose DISCRETE output is the
semantic code the talker consumes. Gated against that module executed directly.
3 cases / 17 assertions.

INDICES ARE GATED EXACTLY: an index off by one codebook entry is a different
utterance, not a small numerical error.

THE NORMALIZATION IS SEARCH-ONLY: distances use L2-normalized encodings and
codebook rows, but decode_code returns the RAW row. Returning the normalized row
leaves every index identical and moves only z_q, so one case pins indices and the
other pins values.

Weight-normed projections carry the legacy weight_g/weight_v spelling; g*v/||v||
is pinned by a hand-computed case.

ONE MUTATION SURVIVED, AND SHOULD HAVE. Removing the encoding normalization is a
provable NO-OP: dist = |e|^2 - 2 e.c + |c|^2, |e|^2 is constant per frame and a
normalized codebook makes |c|^2 = 1, so argmin reduces to argmax(e.c), which
scaling e cannot change. This repo's rule is that an unmoved mutation usually
marks a FIXTURE GAP, so it was checked rather than assumed; it is an algebraic
identity, and the reasoning is recorded at the call site.

Other mutations RED at compile_err=0: returning the normalized row, and dropping
the sqrt from weight-norm.

REMAINING: EnhancedCodec's VocosBackbone encoder, then talker head, length
regulator, S2Mel. No e2e claim.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

… (#715)

FOLLOWING_AGENTS_PROTOCOL

Completes EnhancedCodec: VocosBackbone (ConvNeXt-1D) joins the factorized VQ
already on main. 2 cases / 91 assertions against upstream executed directly.

THE OUTPUT IS [T, dim], not [dim, T] — the final layer norm runs on the
transposed tensor and is never transposed back. Same element count either way, so
the layout is asserted alongside the values.

A TOLERANCE THAT ACCEPTED A WRONG ACTIVATION, found by mutation and fixed in
three steps, each correcting a wrong hypothesis:

1. Swapping exact-erf GELU for tanh-approx PASSED at 2e-5. First guess was small
   activations, so the INPUT scale went 1.0 -> 4.0. No effect: the LayerNorm
   before pwconv1 normalizes input scale away entirely, so the pre-activation
   magnitude is set by the WEIGHTS.
2. Raised pwconv1's weight scale 0.3 -> 1.0, moving |x| from ~0.85 to ~2.8 where
   the GELU forms differ most. Still passed.
3. MEASURED instead of guessing again: the true end-to-end difference is 1.41e-5,
   genuinely below the 2e-5 bound. Tightened to 2e-6, which the correct port
   clears comfortably; the mutation now goes RED.

A bound must be set by what the DEFECT costs, not by what feels safe. Fixture and
tolerance are annotated with the measurement.

Four of six pipeline stages complete. Remaining: talker head, length regulator,
S2Mel. No e2e claim.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…634) (#719)

FOLLOWING_AGENTS_PROTOCOL

S2Mel's DiT conditions on the timestep/style vector through ADAPTIVE LAYER NORM
rather than cross-attention. Ports that mechanism plus the FinalLayer using it.
4 cases / 48 assertions against upstream executed directly.

modulate(x, shift, scale) = x * (1 + scale) + shift. THE `1 +` IS THE DETAIL:
without it the modulation centres on 0 instead of 1, which still trains and still
generates. Pinned by a hand-computed case, not only the oracle — with scale = 0
the result must be the IDENTITY plus shift.

THE FINAL NORM HAS NO AFFINE PARAMETERS (elementwise_affine=False), and the
manifest confirms it: linear.* and adaLN_modulation.1.* exist,
norm_final.weight/bias do not. A case asserts exactly zero-mean, unit-variance
rows.

CHUNK ORDER IS [shift, scale], pinned by effect: zero the modulation weight, set
shift = 0 and scale = -1, and x * (1 + -1) + 0 collapses to zero so the output is
exactly the linear bias. Swapped, it would not be.

The first version of that case asserted identical output ROWS, which was wrong —
layer norm makes rows zero-MEAN, not equal. The oracle comparison passed
throughout, so the failure was the TEST's, and it was rewritten around the
zero-collapse rather than loosened.

MUTATION EVIDENCE with compile status: dropping the `1 +`, swapping chunk order,
and a phantom affine gamma each RED at compile_err=0.

Remaining: S2Mel's DiT attention blocks, the talker head, composition. No e2e
claim.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…at accepted all-zeros on bf16 (#517) (#718)

FOLLOWING_AGENTS_PROTOCOL

W4 of MODEL-TEXT-nemotron-h (#517): the NemotronHForCausalLM hybrid forward --
23 Mamba2, 6 GQA attention at {5,12,19,26,33,42}, 23 non-gated relu^2 MoE.

The implementation was inherited from a rescued WIP whose author's session died
mid-build with nothing compiled and no RED-first evidence, and its gate turned
out to be VACUOUS. On the bf16 arm -- the released checkpoint's dtype -- the
band was 6e-2 ABSOLUTE against an attention reference whose peak is 1.06e-2, so
a block returning all zeros passed. Only the f32 arms were gating anything.

The fresh review proved that by experiment rather than accepting the report: it
reconstructed the inherited gate, made the attention mixer return zeros, and
watched BOTH bf16 arms accept it. Against the repaired gate the same mutant
fails on all four arms. Had this landed as rescued, W4 would have shipped a
suite that could not distinguish a working forward from a dead one in the dtype
the model actually runs.

Repairs: peak-relative bands; every comparison self-certifies by REQUIREing that
its own band rejects all-zeros; the attention fixture was degenerate (logits
~0.09 gives a near-uniform softmax, blind to RoPE, causality AND scale); and the
SSM-dtype guard moved to the stored state, because A = -exp(A_log) decays
downstream separation to 3.16e-5, below the gate's own resolution. 161 -> 254
assertions.

A conflation this operator introduced is corrected. Folding
routed_scaling_factor into the ROUTER is invisible to the model-level gate --
13/13, 254/254, separation 1.91e-07 -- because norm_topk_prob renormalisation
cancels a constant factor, which layer.py:291-300 states outright by forcing the
router factor to 1.0 "so it ends up being a nop". The "13276 of 20000 rows"
figure was always about a DIFFERENT fold, vt::MoeCombine's own semantics, gated
bitwise elsewhere. Spec 6c separates them.

Seams: vt::FusedChain for both residual add+RMSNorm sites (reverting to the
inherited hand-calls REDs check-fusion-consistency); ModelRegistry::Forward.
dense_attn::AttnBlock is recorded NOT APPLICABLE rather than skipped -- it needs
PagedKvCache, slot_mapping, block_table and loader-built weights, the same
boundary kimi_linear_forward.cpp sits on, and W6 owns it. The forward composes
vt::Attention rather than hand-rolling a parallel path.

15 mutations all RED, the table reproducing numerically under independent
re-application. ctest 433/433, clean -Werror, Debug arm identical, preflight
green. Reproduced on Thor sm_110 with CUTLASS correctly disabled: forward
13/254, scaffold 12/38285, Mamba2 suites matching GB10 exactly. That proves a
CUDA-enabled build on a second architecture and NOT GPU execution of the
forward, which asserts a CPU queue by design -- a caveat drawn in the spec and
never upgraded.

Owed and stated plainly: there is no weight loader, so nothing materialises the
18487 tensors, no checkpoint runs, and the forward refuses by name. W5, W6 and
W7 remain. No speed claim. Row stays INVENTORIED.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…#382 (#564)

Commits the spec for the ROCm head_dim=128 decode arm, the ROCm half of #382.
Zero source files; the implementation is not merged and this spec says so
explicitly.

Every code and upstream anchor it cites was verified exact at the pinned
oracle during review: bf16_decode_opt at rocm_paged_attn.hip:1684 matches its
quoted snippet verbatim, the EPL static_asserts at 268/289, the launch switches
at 1875-1904 with only 8/16 instantiations, the five-combination fallback
dispatch at 1937-1947, the CUDA counter-claim at cuda_paged_attn.cu:321/329/2796,
and CALL_CUSTOM_LAUNCHER_BLK_HEAD upstream at the pin.

Review repairs landed on the branch before merge. The blocking one: the spec
opened with "Landed the ROCm d=128 decode arm" when nothing had landed --
git log -S'VT_ATTN_DECODE_D128' -- src/vt/rocm/ is empty on main and
rocm_paged_attn.hip still gates on d == 256 || d == 512. Merging that text
would have put a false "landed" on main, where the next agent greps for the
flag, finds nothing, and cannot tell "never merged" from "reverted" from
"renamed". Renamed to "Result on the implementation branch" with a banner
quoting the two commands that show it, and section 4 moved to future tense for
the same reason.

Also added the two sections AGENTS.md requires and the spec lacked -- Risks and
decisions, and Stop conditions -- drawn from material already in the spec
rather than invented: the bf16-tie reduction-order risk that is why the arm
ships default OFF, the single-board provenance of the 3.53x, the sm_110
1.6x-SLOWER reversal recorded but deliberately unreconciled, the dangling
VT_ATTN_DECODE_WMMA forward reference, and the unquantified residual #488 gap.
Plus a base-SHA mismatch and an upstream line anchor that had drifted by two.

The spec is honest about what it could not gate: it labels its own throughput
table "indicative, not the 2-3x-idle-reproduced standard" and states that no
post-change per-call oracle re-measure was run.

Known-baseline failures only: windows-msvc-* are the PR-only arm (#584), and
sanitize-cpu (address,undefined) is red on main itself (#674, fix pending in
#688). This PR changes only Markdown.

…, rotary (#634) (#721)

FOLLOWING_AGENTS_PROTOCOL

The DiT's transformer is gpt-fast's: every block is conditioned by an
AdaptiveLayerNorm around an RMSNorm with rotary position embeddings. Gated
against gpt-fast executed directly. 5 cases / 30 assertions.

TWO CONVENTIONS THAT DIFFER FROM THEIR NEIGHBOURS IN THE SAME MODEL:

  * This AdaptiveLayerNorm is `weight * norm(x) + bias` — NO `1 +`, unlike
    adaln::Modulate used by the FinalLayer and already on main. Two adaLN
    conventions coexist; porting one over the other is silent. The mutation
    applying the FinalLayer's convention here goes RED.
  * The rotary pairs ADJACENT components, not halves. Half-split is more common
    and yields a rotation that is smooth, norm-preserving and wrong.

RMSNORM DOES NOT SUBTRACT THE MEAN, pinned structurally as well as against the
oracle: for a row of identical positive values x / rms(x) = 1 everywhere, where
LayerNorm gives 0.

The rotary gets a value comparison AND a norm-preservation property, together —
a rotation preserves pair magnitude under EITHER pairing, so the property alone
cannot distinguish them.

FIDELITY NOTE: precompute_freqs_cis returns BFLOAT16, so the rotary table carries
~3 decimal digits before any rotation runs; the tolerance reflects that.

MUTATION EVIDENCE with compile status: wrong adaLN convention and half-split
pairing each RED at compile_err=0. A comment-only NO-OP control ran in the same
batch and stayed green, showing the harness is not simply failing on any edit.

Remaining: assembling the S2Mel block, the talker head, composition. No e2e
claim.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

FOLLOWING_AGENTS_PROTOCOL

Composes the primitives on main into gpt-fast's TransformerBlock as the DiT uses
it, gated WHOLE against upstream executed directly. 7 cases / 34 assertions.

  h   = x + attention(attention_norm(x, c))
  out = h + feed_forward(ffn_norm(h, c))

BOTH RESIDUALS ARE FULL, unlike the macaron HALVES in the w2v-bert Conformer
already on main — two block types with different residual conventions in one
lane. The halving mutation goes RED. SwiGLU gates on W1 not W3, and rotary
applies to q and k but NEVER to v.

A GENERATOR BUG THE GATE CAUGHT: two cases began failing with ZERO failed
assertions — they THREW rather than asserted, because kQ was emitted at the wrong
size. Cause: the new block section looped `for n, q in ff.named_parameters()`,
SHADOWING the rotary input q. Reading only the assertion line would have shown
"12 passed | 0 failed" and looked fine; the CASE count exposed it, and after the
fix assertions went 12 -> 34 because ~22 had been skipped. This is the repo's
recorded `grep assertions:` trap met from the generator side, so mutation results
here are reported by CASE COUNT.

MUTATION EVIDENCE with compile status: swapping W1/W3, skipping rotary on K, and
halving the attention residual each RED at compile_err=0.

S2Mel's block is complete. Remaining: the talker head, then composition. No e2e
claim.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…now has gated numerics (#634) (#724)

FOLLOWING_AGENTS_PROTOCOL

The talker (UnifiedVoice, model_v2.py) runs the GPT-2 backbone already on main
over a sequence assembled from text tokens, mel codes and a speaker latent. This
ports that assembly. 5 cases / 46 assertions.

ITS POSITION EMBEDDINGS ARE NOT GPT-2's wpe: LearnedPositionEmbeddings is a
SEPARATE learned table added before the backbone runs, so the backbone's own wpe
applies as well. Treating them as one silently halves the positional signal.

get_fixed_embedding(ind) is the INCREMENTAL-DECODE path: at step n the position
is n. Returning row 0 every step makes every generated frame believe it is the
first, which still decodes to audio and destroys the prosody.

NO GOLDEN FILE, deliberately: this layer is pure INDEXING and a captured tensor
would only restate its own table. The fixture builds a table whose row r holds
r*10 + d so a row's identity is readable from any element, and the cases assert
WHICH row was read. Capacity is bounded, since
text_pos_embedding.emb.num_embeddings is the talker's real request limit.

MUTATION EVIDENCE with compile status, by CASE COUNT: always returning row 0,
dropping the capacity bound, and dropping the position add each RED at
compile_err=0.

MILESTONE: every one of the six pipeline stages now has its numerics ported and
gated against upstream. What remains is different in KIND — composition, the
checkpoint loader, and only then a render. No e2e claim; the lane still refuses
by name.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…yout the gate caught (#634) (#725)

FOLLOWING_AGENTS_PROTOCOL

Wires the six gated stages into ONE path in upstream's order
(infer_v2_5.py:569-660) and gates the composition. 8 cases / 433 assertions.

WHAT THIS IS: a STRUCTURAL composition at reduced dimensions on synthetic
weights. It proves the stages connect, each output shape is the next input shape,
a change at the front reaches the back, and a mismatch throws BY STAGE. NOT a
quality result, NOT parity, NOT a render.

A REAL BUG THE GATE CAUGHT: style_in was built frame-major (T, F) and handed to
StatsPool, which expects CHANNEL-MAJOR [C, T], so the pooled "channels" mixed
features with frames. Every shape assertion passed and the statistics looked
plausible.

It surfaced because TWO MUTATIONS SURVIVED the first version of this gate —
constant prompt condition, and codec bypass — since propagation cases only prove
SOME path carries influence, not that each stage's output is consumed. Two
isolation cases fixed it: a TIME-PERMUTED clip has identical per-channel
statistics so the style must be unchanged while the condition reorders (which is
what exposed the transpose, since the codes rotate with identical multisets); and
nearest interpolation COPIES samples, so every prompt-condition value must appear
verbatim in the QUANTIZED tensor.

Fixture notes: the first permutation used a REVERSAL, but this fixture's codes
are palindromic (8 11 8 8 11 8 8 11 8) so reversing is a genuine no-op — a
rotation preserves statistics just as exactly and does reorder. The transposed
layout is now itself a mutation, so the bug cannot return unnoticed.

MUTATION EVIDENCE with compile status, by CASE COUNT: constant prompt condition,
codec bypass and transposed style layout each RED at compile_err=0.

STATE: all six stages ported, gated and composed. Before any render: the
CHECKPOINT LOADER and the vLLM-Omni oracle (#633). The lane still refuses by
name.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

… the ACTIVE contract (#672) (#723)

W0 is done and W1 has landed, so the row's state was stale at `SPIKE` while real
loader code sat in the tree. Advancing it is what surfaced two things worth
recording rather than quietly fixing.

THE ACTIVE CONTRACT IS ENFORCED, AND THE SPEC DID NOT MEET IT. Moving the row to
`ACTIVE` turned check-agent-record red on nine missing structured fields -- Scope,
Upstream chain, Our baseline, Port map, Tests to port, Gates, Dependencies, Work
breakdown, Risks/decisions. A `SPIKE` spec is not held to that; an `ACTIVE` one is,
which is the checker doing exactly its job. My first attempt added them as PROSE
SECTIONS and stayed red on all nine, including the two the spec already had --
because the checker does not read headings at all, it reads TABLE ROWS whose first
cell names the field. Fixed by writing the structured record in the shape
`minimax-h3.md` already uses, rather than by arguing with the gate. Worth stating
plainly: the failure was mine reading the requirement as prose, and the checker's
message was right both times.

SECOND, A WORDING FIX THAT COST TWO IMPLEMENTERS. §3 said both pins "go into
`.agents/oracles/` in W0", which reads as future work. `sglang-omni.md` has existed
since #679, yet two separate agents reported it as still owed -- the same agent
twice. A record surface described in the future tense reads as a TODO no matter
what the tree holds, so it is now present tense with links, plus a note saying why.
That is a small edit that stops a recurring false report.

WHAT THE ROW NOW CLAIMS, and nothing beyond it: oracle gateable with 13 committed
per-stage goldens (#708), modular loader gated 1413/1413 against the real 27 GB
checkpoint with all 1012 tensors accounted and the native arm refused by name
(#714). Checklist mark 📋 -> 🚧; rollup SPIKE 9 -> 8, ACTIVE 9 -> 10.

The lifecycle change owes STATUS, BENCHMARKS and the spec's `## Now`, and all
three say the same thing the row does. BENCHMARKS records every axis PENDING with
NONE OWED YET and says why: W0/W1 are correctness scaffolding, the oracle capture
was CPU-only because dgx.casa was down, and there is no forward pass to time. A
row that has landed code is exactly where a speed claim tends to appear by
implication, so the absence is written down rather than left to be inferred.

Both public rows were first written over the house-style limits (a 402-char
BENCHMARKS cell, and STATUS at 45 oversized cells against a 44 ratchet that may
only shrink). Shortened to fit rather than lowering a ratchet to accommodate
prose.

Issue: #672

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…from the shipped checkpoint (#634) (#727)

FOLLOWING_AGENTS_PROTOCOL

Reads IndexTTS-2.5's own config.yaml and repository manifest instead of the
recipe page, pins the dimensions in indextts2_config.h, and gates them against a
COMMITTED copy of the config so the check is reproducible offline. 4 cases.

FOUR FINDINGS — two settle open questions, two ADD scope:

THE MODEL RUNS AT TWO SAMPLE RATES. Talker mel front end 24 kHz / 100 bins;
S2Mel and vocoder 22.05 kHz / 80 bins; OUTPUT is 22.05 kHz. Conflating them
produces audio at the wrong SPEED rather than an error.

THE LANGUAGE QUESTION IS SETTLED AND THE RECIPE PAGE WAS WRONG: the shipped
tokenizer is multilingual_zh_ja_yue_char_del.tiktoken — zh, ja, yue. This spec
had flagged the disagreement as unresolved. Nothing may claim Spanish or Arabic.

THE TOKENIZER IS TIKTOKEN, with no tokenizer.json — the constraint already
recorded for Kimi-Linear.

A QWEN-0.6B EMOTION MODEL SHIPS INSIDE THE CHECKPOINT (qwen0.6bemo4-merge/ with
its own safetensors, tokenizer and config), plus feat1/feat2 matrices and an
emo_condition_module. A second language model inside a TTS lane, NOT in this
spec's scope and covered by no work item.

Also: BigVGAN and w2v-bert-2.0 are NOT in this repository; they fetch separately
into hf_cache/ at first run.

No e2e claim. This is the contract a checkpoint loader is built AGAINST.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…686)

FOLLOWING_AGENTS_PROTOCOL

The residual list grouped "multiple images" with video parts and *_embeds under
"the parse tolerates them but they are not routed". That is accurate for video,
audio and the embeds spellings -- nothing is sent, and the user notices. It is
NOT accurate for multiple images: chat_mm.cpp:256-262 locates the FIRST image_url
part and breaks, so a three-image request is SERVED WITH ONE, no error, no
warning.

The distinction is the whole point. "Not routed" implies a visible failure;
truncation is the case where the user gets a confident answer about a subset of
what they sent and cannot tell. AGENTS.md requires an unimplemented arm to be
"refused with a message naming the missing piece", and upstream does exactly that
-- validate_num_items (context.py:409-428) raises VLLMValidationError.

The line is SPLIT rather than reworded, because the two halves now have different
states: video/audio/embeds remain not-routed; images are truncated and tracked as
#686. #607's L1 has since ported the refusal
(BaseProcessingInfo::ValidateNumItems), so wiring this call site to it is L2 --
which makes #686 the reason L2 exists rather than a separate design.

Found while verifying #607 L1, which surfaced the prior art it inherits.

Records only: one markdown file. Every red on this PR is inherited from main and
tracked -- sanitize-cpu (#730, verified failing on main's own run at a3aa02e),
windows-msvc-* (#648/#503), audit-live-rows (#731, the Music3 ACTIVE row, which
fails on a clean main checkout). A one-file docs diff reaches none of them.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…rd that would have caught my fork was never wired (#634, #728) (#734)

Read the shipped IndexTTS-2.5 checkpoint's `qwen0.6bemo4-merge/model.safetensors`
header by HTTP range request (2 MB, no weights) and its `config.json`. The
result CORRECTS the finding recorded in #727.

That directory was recorded as unscoped work, on the assumption that a second
language model inside a TTS lane implied a second port. It does not. It is 310
BF16 tensors of stock `Qwen3ForCausalLM`: hidden 1024, 28 layers, GQA 16 query
heads over 8 KV heads at head_dim 128, intermediate 3072, vocab 151936,
tie_word_embeddings true. `config.json` names that architecture literally, this
tree registers it in `qwen3_dense.cpp`, and `qwen3_weights.cpp:168` already has
the tying branch that explains the absent `lm_head.weight`. What stays in scope
is the IndexTTS-specific wiring around it: feat1/feat2 and emo_condition_module.

`test_indextts2_emotion_arch_covered.py` pins that reduction against a committed
manifest so it cannot rot in prose. Five mutations, five caught, control green:
a tensor we do not read, an unregistered architecture, lm_head present under
tying, a drifted layer count, o_proj losing its GQA width. No compile step, so
no stale-binary confound.

Wiring those gates into CI immediately found a real defect of my own (#728).
`fvq.cpp` had forked `vocoder1d::MaterializeWeightNorm` -- byte-equivalent, but
without that one's check that `g` carries one magnitude per dim-0 slice. Two
definitions of a fold with one home. `test_vocoder1d_single_home.py` catches it
exactly as designed and reported `MaterializeWeightNorm has 2 definitions`; it
had simply never been added to ci.yml, so it only ran when invoked by hand, as
had `test_indextts2_config_contract.py`. A gate that is not wired is not a gate.
The fork is deleted and its three call sites retarget the single home.

The public projection for this lane was also missing from every earlier merge:
`check-doc-checkpoint.py --commit 73d217d` fails on that already-merged
commit. FEATURES.md moves the TTS row from "not started" to a ◐ that says
"assembled, never run", and USAGE.md records that `/v1/audio/speech` does not
exist, so a server answers 404 at the route table rather than failing at runtime.

Verified locally: test_fvq 17/17, test_indextts2_pipeline 433/433,
test_minimax_music3_loader 1393/1393, all compile_err=0; the three newly wired
Python gates green; check-public-doc-tables and doc-checkpoint --staged green.
audit-live-rows stays red, verified IDENTICAL on clean origin/main and filed as
#733 rather than absorbed here.

Still not done: no checkpoint loader, no render, no route, and any parity claim
needs the vLLM-Omni oracle, still UNPINNED (#633).

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…cope they CORRECT upward (#634) (#735)

Read the tensor manifests of `gpt.pth`, `codec.pth` and `s2mel.pth` without
downloading 4.0 GiB of weights. A torch save file is a ZIP whose small
`*/data.pkl` names every tensor while the gigabytes sit in separate blobs, so
`scripts/read-torch-manifest.py` fetches the central directory and that pickle by
range request and unpickles it with a stub Unpickler. 1712 tensors, no torch,
a few hundred KB. The record is committed.

It CONFIRMS four constants in `indextts2_config.h` from a source independent of
the file they were read from. `test_indextts2_config_contract.py` compares the
header to `config.yaml`; both sides share a source, so it proves transcription,
not truth. This compares the header to the WEIGHTS: kTalkerDim 1280 is
emo_layer's square, kStyleDim 192 is spk_emb_proj's input, kVocosDim 384 and
kVocosIntermediateDim 2048 are the codec decoder's ConvNeXt widths, and
kCodecHiddenSize 1024 is that decoder's input.

It also CORRECTS this morning's correction, upward. #734 recorded the emotion
work as "surrounding wiring: feat1/feat2 and emo_condition_module", reasoning
from the fact that the emotion LANGUAGE model is stock Qwen3. That part holds.
But `gpt.pth` carries two further networks that no work item covers and this
tree does not already have: a Conformer encoder at width 512
(`emo_conditioning_encoder`, 38 patterns, rel-pos MHA with pos_bias_u/v, macaron
FFs, depthwise-15 conv module, a Conv2d-subsampling front end at [512, 261632])
and a Perceiver resampler (`emo_perceiver_encoder`, learned latents, to_q/to_kv/
to_out, GEGLU at 2730). An architecture name settles what a MODEL costs and
settles nothing about what a CHECKPOINT contains.

The same reading names what the reduced-dim gates do not model, all of them
green over a smaller network than the checkpoint holds: a WaveNet conditioning
stack inside the CFM estimator, U-Net skip linears across DiT depth, a SECOND
timestep embedder, the whole conditioning front end, three length-regulator
tensors, `net.gpt_layer`, and the codec encoder. `codec.pth` also ships
`optimizer.state`, so part of its 0.57 GiB is training residue.

Six mutations, six caught, control restored and verified byte-wise: two header
constants moved, the language embedding's 107 rows, a vanished DiT skip_linear,
a shrunken emotion Conformer, and a drifted tensor total. Pure data and Python,
so no stale-binary confound.

Also records that the talker embeds 107 language ids while the tokenizer is
zh/ja/yue, which bears on what this lane may CLAIM and is asserted rather than
noted.

audit-live-rows stays red, identical on clean origin/main, tracked as #733.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

… the backbone prefix, refuse unsupported MoE layouts (#490)

FOLLOWING_AGENTS_PROTOCOL

Qwen/Qwen3.8-2.4T-A95B declares Qwen3_5MoeForCausalLM / model_type qwen3_5_moe_text
-- the same GDN-hybrid MoE backbone we already run token-exact on Qwen3.6-35B-A3B.
head_dim 256, linear k/v head dim 128, full_attention_interval 4,
partial_rotary_factor 0.25, rope_theta 1e7, mtp 1 and vocab 248320 all match; the
differences are pure scale and all config-driven.

WHAT THIS DELIVERS: both architecture strings dispatch; the backbone weight
namespace resolves for either spelling; mixed and empty indices are refused; an
unsupported MoE expert layout is refused with a message naming the missing piece;
the dense arm routes BF16/FP8/NVFP4 by tensor presence; the gated rows stay
byte-identical.

WHAT IT DOES NOT DELIVER, AND A CORRECTION. An earlier revision of this row
asserted that the tensor names being "identical modulo the prefix" meant the
loader body was already correct. The names are identical; the conclusion did not
follow. LoadQwen3_5Moe -> LoadMoeExpertsInto -> LoadNvfp4Raw is a QUANTIZED
checkpoint loader: it requires per-expert .weight U8 + .weight_scale F8_E4M3 +
.weight_scale_2. The published bf16 checkpoint has 3-D STACKED experts and ZERO
scale tensors, so it dies at lm_head before reaching them. This was invisible
because our gated 35B row loads the requantized nvidia/Qwen3.6-35B-A3B-NVFP4, not
a published Qwen bf16 repo -- the MoE loader has never read one.

That false claim had reached the spec, a shipped header, docs/STATUS.md,
model-matrix.md and three commit messages. It is struck everywhere, the debt is
recorded the right way round (the quantized arm is the one implemented; the
bf16/3-D-stacked arm is OWED, porting-inventory.md deviation 17(e)), and the
previously-false closing condition is corrected on all seven surfaces. The
dense-vs-MoE asymmetry is stated wherever the limitation appears.

AHEAD OF THE PIN, DELIBERATELY. The parity pin 555967922 has only the
ForConditionalGeneration entries; the text-only arms arrived upstream in vllm#50210
/ ad5d29db7, post-pin. A forward port of one upstream PR; it does not advance the
pin. One tracked divergence: a mixed index is REFUSED here where upstream
WeightsMapper normalizes.

EVIDENCE. CPU: clean -Werror build, serial ctest 431/431, test_qwen3_8_text_only
7 cases / 747 assertions. GPU (dgx GB10 sm_121a) at the reviewed head 1cc951c:
SACRED test_qwen36_paged_engine 315/315, test_qwen27_paged_engine 235/235,
test_qwen3coder_paged_engine 138/138 -- real runs, zero skips, goldens
byte-identical. test_qwen35_plain_weights was a SKIP (4 assertions, Qwen/Qwen3.5-4B
not cached) and is not counted as coverage.

Reviewed independently four times. Reviews caught, and this row fixes: three MoE
prefix mutations that survived the full 396-test suite; a fixture that was not the
published config; the inverted debt record; and a latent UBSan misaligned-load
class (#627) that the new test exposed in the shared loader -- fixed by adopting
the existing vt::LoadUnaligned seam rather than a private copy.

OWED, RECORDED NOT WAIVED: the bf16 / 3-D-stacked MoE expert arm; a re-run of the
SACRED gates against this final head (they were captured at 1cc951c and the
merges since changed no row source -- an argument, not a run); and the 2.4T run
gate, unmeetable at ~4.8 TB bf16 against 128 GB unified. Nothing here claims a
generated token from that checkpoint.

CI: the red windows-msvc lanes (#646) and sanitize-cpu address,undefined
(ltx2_loader.cpp misaligned load, #627) are pre-existing on main and reproduce on
six other open PRs; neither is attributable to this row.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…ge the manifest found missing (#634) (#738)

`s2mel.pth` carries `net.cfm.estimator.wavenet.*` and the shipped config sets
`s2mel.DiT.final_layer_type: wavenet`, so this stack is on the render path. It
was absent from the port map until #735 read the checkpoint manifest, and the
CFM/DiT suites were green over a network without it.

Reading upstream also explains three other names #735 listed separately.
`t_embedder2`, `conv1` and `conv2` are not loose ends: they are the rest of this
same final layer, which owns a second timestep embedder at the wavenet width
(`diffusion_transformer.py:145-153`, index-tts @4f8792ff). The spec's table now
says so instead of listing them as four unrelated gaps.

Ported from `indextts/s2mel/modules/wavenet.py` class WN. Three details the gate
holds, each of which is a plausible wrong guess:

- the convolutions are upstream `SConv1d`, whose default `pad_mode` is REFLECT.
  Zero padding is the natural assumption and it is wrong.
- ONE conditioning projection is computed outside the loop and each layer reads
  its own `2 * hidden` window of it; recomputing per layer, or reading layer 0's
  window every time, both look reasonable and both change the output.
- the mask multiplies the residual update and the final output but NOT the input
  to `in_layers`, so a masked run is not a shorter run. A golden with three
  trailing zero frames pins that asymmetry, and asserts the kept frames DIFFER
  from the unmasked run, since the mask feeds back through the residual.

The weight-norm fold calls `vocoder1d::MaterializeWeightNorm`, the single home
restored in #734, rather than growing a third copy.

RED first: the body shipped as a `VT_CHECK(false, ...)` so the failure was the
unimplemented stage rather than an ambiguous missing-header compile error, and
`compile_err=0` was recorded next to it. GREEN: 3 cases, 133 assertions.

Six mutations, six caught, all with compile_err=0, control restored and verified
by md5: reflect padding to zero, the tanh and sigmoid halves swapped, the
conditioning slice pinned to layer 0, the residual left unmasked, the skip half
read from the residual half, and the output left unmasked. A seventh attempt was
VOID twice before it compiled -- once because the mutation left `ReflectIndex`
unused under -Werror, once because shell escaping corrupted the edit -- and both
times the stale binary printed SUCCESS.

test_cpu_x86_llamacpp_floor fails as NO_QUIET_WINDOW at load 113 from this
session's own builds, not from this change. audit-live-rows stays red, tracked
as #733.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…idth coupling upstream hides (#634) (#739)

Ports `diffusion_transformer.py:243-253` (index-tts @4f8792ff): the long skip,
conv1, the second timestep embedder, the wavenet, the long residual, the final
adaLN layer and conv2. With #738 this makes the DiT's whole post-transformer
path complete, and it CLOSES four separate entries the manifest had listed as
unrelated gaps: `skip_linear`, `t_embedder2`, `conv1` and `conv2` are all one
final layer.

It composes `wavenet::Forward`, `cfm::TimestepFeatures` and `adaln::FinalLayer`
rather than reimplementing any of them.

Building the oracle found a coupling that the shipped config HIDES. `final_layer`
is constructed at the wavenet width but conditioned on `t1`, which the DiT
embeds at its own hidden width, so the wavenet final-layer path only composes
when the two are equal. Both are 512 upstream, so nothing there ever exercises
it; setting them differently in the generator raised
`mat1 and mat2 shapes cannot be multiplied (1x8 and 6x12)` from upstream's own
module. We now refuse unequal widths with a message naming which two numbers
disagree, rather than inheriting a silent coincidence, and a case asserts the
refusal.

The oracle constructs the real upstream DiT at reduced dims and drives its own
module objects, so only the SEQUENCE is restated -- once, beside the upstream
line numbers it copies.

Gated 4 cases / 33 assertions, and the three behavioural cases are chosen to be
discriminating rather than merely non-vacuous: perturbing ONLY `x` must move the
output (without the concat the tail is a function of `x_res` alone and still
produces a mel of the right shape), `t1` must move it with the raw timestep held
fixed, and the raw timestep must move it through `t_embedder2`.

Six mutations, six caught, all compile_err=0, control restored and verified by
md5: the long skip dropping x, res_projection swapped for conv1, the wavenet
conditioned on t1 instead of t2, the transpose into the wavenet made a no-op,
SiLU dropped from t_embedder2, and the output emitted frame-major.

Still unported on this stage: `layers.N.skip_in_linear`, the PER-LAYER U-Net
skip across DiT depth (`uvit_skip_connection: true`), and the conditioning front
end. No render yet.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

… upstream rather than read off the formula (#634) (#741)

`uvit_skip_connection: true` in the shipped config makes the first half of the
transformer push its output onto a stack and the second half pop one, and the
checkpoint carries a `layers.N.skip_in_linear` per receiving layer. With #738
and #739 this completes the DiT path.

The routing is pure index logic, and every plausible misreading still produces a
model that runs: FIFO instead of LIFO, `>=` instead of `>`, pushing a layer's
INPUT instead of its output. So it was not derived by reading
`gpt_fast/model.py:154-190`. `scripts/gen-dit-skip-schedule.py` drives
upstream's own Transformer with the layers instrumented and records which layer
actually received which layer's output, at six depths. The gate compares against
those recordings, which are quoted in the test with their provenance.

At the shipped depth 13: layers 0-5 emit, 7-12 receive, layer 7 takes layer 5's
output and layer 12 takes layer 0's, and layer 6 in the middle does neither.

The recording also found an upstream ASYMMETRY worth keeping. Emitters test
`i < n/2` and receivers `i > n/2`, so at EVEN depth there is one more emitter
than receiver and the earliest skip is pushed and never popped -- at depth 12,
layer 0's output is simply discarded. `Schedule::orphaned` reports it. A port
that "fixed" this would be a different model, so it is preserved and asserted
rather than corrected.

The concatenation order is pinned by hand-computed cases rather than a golden:
`cat([x, skip])` reads the two halves with distinguishable multipliers, and the
reversed call is asserted to give a DIFFERENT answer, so the order is proven
rather than merely exercised.

RED first, all six cases throwing on an unimplemented body with compile_err=0.
GREEN 6 cases / 52 assertions. Seven mutations, seven caught, all compile_err=0,
tree verified byte-identical afterwards: `>=` for receivers in both the list
build and the routing loop, `<=` for emitters, FIFO instead of LIFO, emitting
before the layer instead of after, the skip half reading x's columns, and the
orphan count suppressed. Two earlier attempts were reported VOID by the harness
on a non-unique and an absent anchor rather than silently mutating a neighbour.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…s with POSIX stat

FOLLOWING_AGENTS_PROTOCOL

`src/vllm/multimodal/video_engine.cpp` reached Windows with unguarded POSIX
`stat` -- `:21` `#include <sys/stat.h>`, `:57-60` `IsDir`, `:62-65` `Exists` --
which `check-windows-portability.py:1675-1688` flags under `full_source_posix`,
i.e. every scanned source and not only the platform-boundary set. It reddened
`windows-msvc-cpu` and `windows-msvc-vulkan` on EVERY open pull request in the
repository, across five unrelated lanes. It hid because those jobs are
`skipped` on `main` pushes (#584), so a green main carried no information.

Repaired at the SOURCE, never the checker: `IsDir`/`Exists` now take the
`std::error_code` <filesystem> overloads through a file-local path helper,
preserving `::stat`'s return-false-for-an-uninspectable-path behaviour that the
THROWING overloads would have turned into a `filesystem_error` escaping a
registry query. The helper adopts `fs_io.cpp:31`'s spelling byte-for-byte and
deliberately NOT `minimax_h3_sharded.cpp:55`'s, which throws on malformed UTF-8
under _WIN32 -- that would reintroduce on Windows the exception this removes on
POSIX. The wider divergence is #687.

RED first, and the honest RED is worth recording: a missing-path test would
have passed with and without the guarantee, because ENOENT does not throw. The
real probe is a 300-character path component (ENAMETOOLONG).

Reviewed PASS by a fresh reviewer who reproduced the premise from another
lane's CI log, confirmed by sha256 that the checker was untouched, and REJECTED
the implementer's probe for the one surviving mutation -- "fails to refute is
not unreachability" -- then established unreachability from the library
mechanism instead: `IsDir` is TU-local with one call site gated by `Exists`,
and libstdc++ implements both via `f(status(p))`, so `exists()` can only return
true when the stat already succeeded.

Operator gate at the merged tree: WINPORT_EXIT=0, BUILD_EXIT=0, 0 errors,
0 ENOSPC, REGISTERED=444, test_video_engine 12 cases / 260 assertions exit 0.
Two ctest failures both attributed by measurement, not assumption:
`test_op_parity` is #737 (a null in the Music3 golden added today -- proven by
moving that one golden aside: 10/10 pass, assertions 70 -> 123), and
`test_openai_conformance` is a load artifact at load average 115 that passes
alone, exit 0, 48 assertions.

This does NOT turn the Windows lane green on its own. Two independent defects
were stacked; with this one gone the build reaches `M_PI`, which MSVC does not
define -- #720, fixed by #732.

Closes #664.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5[1m] [claude-code]

…s::pi_v<double>

FOLLOWING_AGENTS_PROTOCOL

The SECOND of two stacked Windows defects, both from the LTX-2.5 lane. #664
(landed as f8cbc23) removed the POSIX `stat` one; with that gone the MSVC
build reaches `M_PI`, a POSIX extension that MSVC's <cmath> defines only behind
`_USE_MATH_DEFINES`:

  ltx2.cpp(72,58) / (557,69) / (572,76): error C2065: 'M_PI': undeclared identifier
  ltx2.cpp(72,15): error C2737: 'kBeta': const object must be initialized

Those four are the ENTIRE `error C` set in both job logs.

The issue's own scope was wrong in both directions, and the implementer
corrected it with a positive control in the same grep. `git grep -- src include`
never looked in tests/, so it MISSED `tests/vllm/models/test_vocoder1d.cpp`,
which also uses M_PI unguarded; and it counted `ltx2_audio_vae.cpp` and
`ltx2_video_vae.cpp`, which compile fine behind their own `#ifndef M_PI` (the
latter's define being dead -- zero uses). All four sites now use
`std::numbers::pi_v<double>` and the two hand-rolled defines are gone; extra
spellings of one constant are #687's disease.

The gate for this ALREADY EXISTED and was ALREADY RED at base, naming all four
files: `test_check_windows_portability.py` asserts no `\bM_PI\b` under src/ or
tests/. It never fired because that suite runs in NO workflow (#680, #646). So
LTX-2.5 landed M_PI on top of a gate that was already objecting and nothing was
listening -- which, with #584, is why two Windows defects stacked unseen.

MSVC's header condition was reproduced on Linux with
`-U_GNU_SOURCE -D_ISOC99_SOURCE`, reporting ltx2.cpp at the same LINE AND
COLUMN as the real CI log, and nothing for the two guarded VAEs -- confirming
their guards by measurement. Stated precisely as a g++/glibc proxy, not MSVC.
Independent corroboration on the target toolchain: MSVC compiled
`vocoder1d.cpp`, `minimax_h3_video_vae.cpp` and `mla_attention.cpp` cleanly in
the same log, and all three already use `std::numbers::pi_v<double>`.

Value identity established three ways: bit pattern 400921fb54442d18 for all
three spellings and for the derived sqrt(2/pi) and pi/2 in float and double;
token-sequence equality over 41,191 tokens; and 7 golden suites, 185 cases /
17,378 assertions, identical in both arms.

Reviewed PASS by a fresh reviewer whose independently re-derived TU list
matched, who verified the line-and-column match against the job log itself, and
who caught a real overstatement: `test_vocoder1d.cpp` is NOT a lane blocker,
because `build-windows-release.ps1:263-273` builds a fixed focused target list
that never reaches it.

Operator gate: M_PI hits 0 tree-wide (positive control: 8 std::numbers files),
BUILD_EXIT=0, 0 errors, 0 ENOSPC, REGISTERED=443, windows-portability suite
71 passed / 1 failed -- the survivor being #680's, red at base too. The one
ctest failure is #737, main's own Music3 golden red.

Honest limit: after this and #664 the lane has never gotten past the LIBRARY
compile, so link, the eight focused test executables, packaging and the smoke
run remain unproven on Windows. This removes the entire remaining `error C`
set visible in both job logs; it does not prove Windows is green.

Closes #720.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5[1m] [claude-code]

… to read pickle in the engine (#634) (#742)

Upstream ships `.pth`: a ZIP around a Python pickle. Loading it needs a reader
this tree does not have, and the answer is not to write one. Pickle executes
arbitrary code by construction, so a reader in the engine would run an
attacker-controllable program inside the process that serves users, and every
other lane here already loads safetensors or GGUF. The conversion therefore
happens offline, once, and the engine loads the result through the reader it
already has.

`scripts/convert-indextts2-checkpoint.py` flattens the nested state dicts with
'.', which is exactly the naming `indextts2_pth_manifest.json` records, so the
converted names ARE the manifest's names and the manifest can check the
conversion. Run against the real checkpoint on the NAS, all three files matched
their recorded tensor counts: 456, 972, 284.

It also measured something the manifest could only hint at. `codec.pth` is
**75% optimizer state** -- 729 of its 972 tensors are training residue -- and
dropping it takes that file from 579.16 MiB to 192.99 MiB. The drop is reported
with a count, never silent, and the prefix is gated from BOTH sides: every
optimizer key in the manifest must match it, and no weight in `gpt.pth` or
`s2mel.pth` may. A prefix widened from "optimizer." to "opt" would quietly eat
model weights, which is exactly the mutation that proves the second check.

The conversion needs torch and 4 GiB of weights, so CI cannot run it.
`tests/scripts/test_indextts2_convert.py` holds the part where a silent mistake
is unrecoverable -- which tensors survive, under which names -- using fakes,
with neither torch nor the checkpoint, because a dropped weight looks exactly
like a weight that was never there. It also asserts the flattener survives the
ints, strings and Nones that sit beside weights in a `.pth`.

8 cases. Five mutations, five caught, script verified byte-identical after:
the drop prefix widened to "opt", the drop prefix emptied, the flattener joining
with '/' instead of '.', the flattener crashing on non-tensor leaves, and one
source silently omitted.

This unblocks the checkpoint loader; it is not the loader, and there is still no
render.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…er we mostly own on hardware we do not (#699) (#705)

`dots-studio/dots3-note-prev` is a 280B-total / 16B-active multimodal MoE:
text + image + video + audio understanding, 512K context, an MTP head. vLLM
added it on `main` in vllm#51255 (`vllm/models/dots3_note/`, 15 files, ~5.7k
LoC) and was still patching it the day before this scope (vllm#52172,
2026-08-13).

W0 ONLY. Spec committed before implementation, per AGENTS.md. No src/, include/
or tests/ change is in scope; W1 onward are dispatched to fresh implementers
from this spec rather than written in the coordinating session.

THE PORT IS CHEAPER THAN THE PARAMETER COUNT SUGGESTS, because upstream builds
the text tower by subclassing DeepSeek: `Dots3NoteModel(DeepseekV32Model)`,
`Dots3NoteDecoderLayer(DeepseekV32DecoderLayer)`, `Dots3NoteMoE(DeepseekV2MoE)`,
`Dots3NoteFullAttention(DeepseekV2MLAAttention)`. Our gated MLA (deepseek_v2.cpp,
SACRED 8/8), the DSA lightning indexer (deepseek_v4_dsa.cpp), the noaux_tc
sigmoid router, the 256-expert grouped MoE, qwen3_vl_vision.cpp (image+video
STRICT 32/32) and voxtral.cpp (audio 16/16) already carry most of it.

WHAT IS ACTUALLY NEW, and the first item is the campaign:

  * Sliding-window MLA. 33 of 46 layers are SWA (window 513) over MLA with a
    SECOND geometry: 64 heads, swa_kv_lora_rank 1024, swa_qk_nope_head_dim 192,
    own rope_theta 5e4, is_neox_style=False. Upstream needed two Triton kernels
    plus its own MLA metadata builder. Every MLA path we have is full attention
    over a paged cache (deepseek_v4_registry.cpp sets is_hybrid=false).
  * A heterogeneous MLA KV spec: latent rows 576 (full) vs 1088 (SWA), which
    upstream reconciles by padding the physical row.
  * The headwise sigmoid attention gate, an extra RMSNorm over the 64-dim
    rope-only k slice, and the two apply_mla_qkv_lora_rescale scalars.
  * A pyramid MoE ViT (all our vision towers are dense) and the `dots` audio
    stem (conv2d stem, RoPE, RMSNorm, SwiGLU, 6000 positions, 60s chunking).
  * The GGUF k-quant arm, which is owed and has no prior art: llama.cpp has no
    dots3_note, so the converter is ours and there is no quant-matched
    llama.cpp bar.

FOUR CONFIG TRAPS THE CHECKPOINT DOES NOT CARRY, all set in
Dots3NoteConfig.__init__ and all numerically silent. n_group=1 / topk_group=1,
where upstream's own comment says inheriting DeepSeek-V3's 8/4 grouping
"changes the selected experts at every MoE layer" and our router is gated at
V3's grouped dims; indexer_rope_interleave=True (GPT-J adjacent pairs) where
V3.2 and therefore our indexer default to split-half NeoX;
num_nextn_predict_layers=1. A port reading only config.json gets all of them
wrong and still emits plausible text.

TWO BLOCKERS, RECORDED RATHER THAN WORKED AROUND.

Beyond-pin: our parity pin is 555967922 (0.26.0.dev0) and the checkout at that
SHA carries only dots_ocr.py, verified. Gating needs a pin advance, against an
implementation upstream is still changing.

Oracle-memory-infeasible, and this is the decisive one. Measured via the HF API:
131 language shards + model-vision.safetensors 13.7 GB + model-audio.safetensors
1.77 GB = ~576 GB bf16; the fp8 sibling ~290 GB; upstream's recipe is 8xH100.
GB10 is ~119 GiB and Thor ~122 GiB RAM with 123 GiB free disk, so the vLLM
oracle cannot run this model here at any published precision, and no smaller
dots3-note exists in the dots-studio org. Same wall deepseek-v4-flash hit at
156.7 GiB, three times worse. Our own arm is plausible at <=2 bpw, but an arm
with no oracle beside it yields no parity number, so every performance axis is
an open gap by construction.

THOR IS DESIGNATED, AND ITS LIMITS ARE MEASURED. Developer direction names
192.168.68.23 as the e2e CUDA host. Probed read-only 2026-08-14: 14 cores,
122 GB RAM, 123 GiB free on /home, but no nvcc, no cmake, no ninja, no venv, and
nvidia-smi refuses under non-interactive ssh. So W0.5 provisions it, and the
spec states plainly that designating a host does not change the arithmetic
above: Thor can carry our low-bit arm, the sm_110 axis and every unit gate, not
a 290 GB oracle.

The spec ends on a three-way decision that is the developer's, not the
implementer's: rent 8xH100, accept unit-gated bricks with the e2e gate recorded
as owed, or park at W0. No ceiling is declared and the gap stays open.

Records: MODEL ratchet 375 -> 377 in check-agent-record.py, bumped because two
rows EXIST (Dots3NoteForCausalLM at SPIKE, Dots3NoteMTPModel at INVENTORIED),
never to make a transition pass; the beyond-pin ratchet 371 -> 373 and the
rollup 375 -> 377 in model-matrix.md; the #699 row in roadmap_v1.md; a
FEATURES.md "inventoried but blocked" line.

AND THE RATCHET COLLISION FIRED, which is worth recording because the record
predicted it. This branch first took 373 -> 375; while it was in review the #490
branch took 373 -> 375 as well, for the two Qwen3.5 text-only arms. BOTH sides
read 375, neither was right, and an auto-merge keeping either would have left
model-matrix.md internally consistent while silently short two real
architectures -- exactly the hazard
test_music3_and_indextts_rows_both_survive_their_collision was written for. The
merged tree holds four new rows, so the number is RE-DERIVED to 377 by counting
the matrix after the merge. Both conflicted keyed records
(model-matrix.md, check-agent-record.py) were reset to origin/main byte-for-byte
and this row's scoped edits reapplied on top, per the keyed-record rule; #490's
two rows are verified present and the only non-dots3 lines that differ from
main are the eleven count lines this change owes.

Gates: check-agent-record OK (MODEL=375), check-model-checklist OK,
check-public-doc-tables OK, test_agent_record 55/55, doc-checkpoint --staged OK,
now-current --staged OK. `agent-preflight.sh --staged` is green except
test_cpu_x86_llamacpp_floor, which is #618's load-dependent NO_QUIET_WINDOW
(exit 4 instead of 2) and REPRODUCES ON A CLEAN TREE AT THE BASE SHA
a3aa02e at loadavg ~50-70. Not caused by this
change, which touches no code that test exercises.

CHECKER CHANGE, WITH THE MUTATION EVIDENCE THE pr-size GATE DEMANDED. The MODEL
ratchet bump is a checker-semantics change, so it now carries
test_dots3_rows_are_inside_the_model_ratchet. Measured, not asserted: RENAMING
the MTP row leaves the count at 375, touches no claim, keeps every other check
green, and REDs only the new assertion -- which is exactly the hazard a count
cannot see. Two further mutations are recorded in the test's docstring as
WEAKER evidence rather than presented as its force: INVENTORIED -> SPIKE is
already caught by the claim-ownership rule, and SPIKE -> ACTIVE by the
structured-spec rule. Every mutation was reverted and model-matrix.md restored
byte-for-byte (md5 5c47ef2c045dd64c3b7b32a2aebf01a1 before and after).

That last mutation also surfaced a constraint worth recording rather than
rediscovering: an ACTIVE row owes this spec seven structured headings (Scope,
Upstream chain, Our baseline, Port map, Tests to port, Gates, Dependencies)
that a SPIKE row does not. Spec section 8.1 now says so, so the W1 lifecycle
move budgets for it.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…ghts (#634) (#743)

First time any IndexTTS stage meets its actual checkpoint.
`indextts2_s2mel_loader` binds the converted `s2mel.safetensors` to the ported
`dit_tail`, and the shipped 512-wide tower runs: 80 x 8 mel, values in
[-1.14, -0.31].

Dimensions are resolved FROM THE WEIGHTS, never from a config: hidden 512 and
in_channels 80 come out of `skip_linear.weight` being [512, 592], the wavenet's
8 layers are counted, kernel 5 is read off `in_layers.0.weight_v`, and
`t_embedder2.mlp.0.weight` being [512, 256] is what says the sinusoidal width is
256. A config disagreeing with the weights would be the config that is wrong.
Three things are recorded rather than assumed: the tower is F32 throughout (a
rare case where upstream really does store f32), `weight_g` ships as [out, 1, 1]
so flattening is asserted rather than presumed, and `dilation_rate` is NOT in
the checkpoint at all -- upstream ships 1, the shapes cannot distinguish it, and
the code says so instead of implying the checkpoint proved it.

MUTATION TESTING FOUND TWO REAL HOLES IN THIS GATE, which is the point of it.
Deleting the dtype check survived, and so did making a missing tensor return an
empty vector instead of throwing. Both passed because the fixtures never
exercised those paths: the "missing tensor" case happened to fail inside the
SHAPE lookup, never the data read, and no case ever fed a non-F32 tensor.

Closing the second was itself two mistakes deep. An I64 tensor throws on BYTE
LENGTH rather than dtype, so it kills the mutation for the wrong reason; and an
otherwise-incomplete fixture throws on the missing tensors instead. The case now
uses a COMPLETE checkpoint with exactly one I32 tensor -- four bytes, exactly
F32's width -- so only the dtype check can catch it. All six mutations caught,
file verified byte-identical.

One more instrument bug worth recording. The real-weights case first reported
mel values of 1e16 and I nearly filed it against the port. The port was fine:
`0.01F * ((i % 37) - 18)` with a size_t `i` is UNSIGNED arithmetic, so every
value below 18 wrapped to ~1.8e19 and the tower was fed 9e16. A stage-isolating
harness showed `x_res` was already enormous before anything ran, and the wavenet
was returning a perfectly sane 0.085. The inputs are now cast and, more to the
point, ASSERTED before anything downstream is believed.

Gated 8 cases: 36 assertions synthetic, 4782 with the real checkpoint attached
via VLLM_CPP_INDEXTTS2_S2MEL. Without that variable the real-checkpoint cases
skip LOUDLY, so "never checked" cannot read as "passed".

This is not a render. The mel here is driven by synthetic transformer output,
because the DiT blocks and the conditioning front end are still unported, and
correctness against vLLM-Omni still needs the oracle pin (#633).

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…r that is DEAD in 2.5 (#634) (#745)

Ports `diffusion_transformer.py:206-226` (index-tts @4f8792ff): cond_projection,
the four-way concatenation, the style broadcast, the classifier-free-guidance
zeroing, and cond_x_merge_linear. The shipped
`cond_x_merge_linear.weight` is [512, 864], and 864 is exactly
512 + 80 * 2 + 192, so the concatenation order and widths are pinned by the
weight itself rather than by my reading of the code.

Two things contradict what a reader expects, and both are now recorded where
someone will hit them.

`cond_in_module` is FORCED to `cond_projection` upstream: the `content_type`
switch that would have selected `cond_embedder` is commented out. So
`cond_embedder` is present in `s2mel.pth` and DEAD in 2.5. A port that
"restored" the switch, which is exactly what reading the surrounding code
suggests, would consult a tensor this model never uses.

`class_dropout` is NOT a training-only path. `mask_content` sets it at
inference, and it zeroes everything AFTER the first `in_channels` columns --
keeping x, dropping prompt, cond and style. That IS the CFG unconditional
branch that `cfm::EulerStepCfg` consumes, so it is ported as a first-class
argument rather than an artefact.

Gated against upstream on BOTH branches, and the behavioural cases are chosen to
separate near-misses rather than merely to be non-vacuous: under the
unconditional branch, perturbing x must move the output while perturbing prompt,
cond or style must NOT; under the conditional branch all four must move it; and
style is proven BROADCAST rather than read per frame by shifting the whole
vector and requiring every frame to move by the same delta.

5 cases / 128 assertions. Six mutations, six caught, all compile_err=0, file
verified byte-identical: raw cond used instead of the projection, the concat
order swapped, x read frame-major instead of channel-major, the unconditional
branch zeroing x as well, the unconditional branch keeping style, and style
indexed by frame instead of broadcast.

Still no render: the DiT BLOCKS between this front end and the ported tail are
not wired, and correctness against vLLM-Omni needs the oracle pin (#633).

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…imator is COMPLETE front to tail (#634) (#748)

Ports `gpt_fast/model.py:161-191` (Transformer.forward): N blocks, the U-Net
skip routing across them, and the final AdaptiveLayerNorm. With #738, #739,
#741 and #745 the S2Mel DiT now exists end to end -- front end, stack, tail --
each gated against upstream's own modules.

This is composition only. The block is `dit::Block`, the routing is
`dit_skip::Plan`, the merge is `dit_skip::ApplySkip`, and nothing here
reimplements any of them. The rotary table is passed IN, emitted by the
generator from upstream's own `freqs_cis`, so the gate measures composition
rather than a second copy of `precompute_freqs_cis`.

One upstream quirk is recorded rather than tidied: a `skip_in_linear` is built
on EVERY layer when `uvit_skip_connection` is set, including layers that never
receive a skip, so the checkpoint carries unused ones. They load and are left
alone. A case asserts that layer 0's skip_in_linear cannot affect the output at
all, while a receiving layer's must.

3 cases / 52 assertions against upstream Transformer.forward.

Five mutations, five caught, all compile_err=0, file byte-identical after: the
skip merged against itself instead of the popped value, emitting a layer's INPUT
instead of its output, FIFO instead of LIFO, the final norm dropped, and the
receive moved after the block.

A SIXTH mutation was withdrawn as badly designed rather than reported as a gate
hole. Pushing before the block WITHOUT removing the push after it leaves the
stack holding both, and because a layer's input IS the previous layer's output,
every pop still returned the same tensor -- a genuine no-op, not a defect the
gate missed. Rewritten to emit the input INSTEAD of the output, it fails.

Still no render: the talker, the reference encoders and BigVGAN are unported, so
nothing yet produces the conditioning this estimator consumes, and correctness
against vLLM-Omni needs the oracle pin (#633).

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

Merges the PINNED SHA `32d82c64d`, not the ref. `origin/main` is shared with
other worktrees in this checkout, and it advanced from `5e646d98c` to
`32d82c64d` mid-merge when another session fetched — which made main's own newer
files (the `dit_front` / `dit_stack` IndexTTS2 cluster, #745/#748) read as
"silently dropped by the merge" in every `git diff origin/main` afterwards. They
were never dropped; they are simply not in `5e646d98c`. That first attempt was
aborted rather than patched, because the repair it invited — restoring files
from the newer main into a tree merged from the older one — produced a
CMakeLists listing TUs the tree did not contain. Merging an immutable SHA is
what makes the verification below mean anything.

Three conflicts, all resolved against that SHA:

* `tests/vllm/multimodal/test_ltx2_video.cpp` — both sides added a different
  include in the same sorted run: this branch's `ltx2_video_vae_encoder.h` and
  main's `vllm/platforms/interface.h`. Both are kept, in sorted order. Proved by
  COMPILING, not by reading: `BUILD_EXIT=0` and the suite runs.
* `.agents/roadmap_v1.md` and `docs/FEATURES.md` — keyed records, resolved by
  taking main's file WHOLESALE and reapplying this row's key onto it, never by
  accepting the automatic three-way result. Each file is now byte-identical to
  `32d82c64d` apart from exactly one line: the `#644` row and the LTX-2.5 DiT
  row. An earlier merge in this campaign auto-merged two `#644` rows into a
  duplicate that `check-agent-record` caught; taking one side wholesale is what
  makes that shape impossible rather than merely unlikely.

The combined `#644` row names both campaign rows and both specs. The FEATURES
row carries main's prompt-AdaLN facts plus this row's served image-conditioning
arm, inside the 220-char cell and 600-char entry budgets.

`docs/USAGE.md`, `CMakeLists.txt` and `tests/CMakeLists.txt` auto-merged, and
were each verified against the pinned SHA rather than trusted: both CMake files
are PURELY ADDITIVE (zero removed lines; this branch adds the two
`ltx2_video_vae_encoder_load` / `ltx2_image_preprocess` TUs and the one
`test_ltx2_image_cond` registration), and USAGE's only removal is the paragraph
this row supersedes, with main's own additions still present.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]